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Chapter  1 


Introduction 

1.1  Background 

Networks  of  coupled  oscillators  have  been  proposed  for  various  applica¬ 
tions,  including  locomotion  (specifically,  central  pattern  generators),  pattern 
recognition  (for  instance,  distinguishing  between  different  objects  in  view  si¬ 
multaneously),  and  biological  information  processing  (as  a  solution  to  the 
dynamic  binding  problem).  There  is  evidence  that  coupled  oscillator  net¬ 
works  play  a  role  in  biological  systems,  but  the  point  of  view  taken  in  this 
work  is  that  the  mathematical  analysis  of  coupled  oscillator  networks  can 
also  yield  networks  which  are  useful  in  their  own  right  for  applications  in 
pattern  recognition,  control,  and  communications. 

Experiments  on  the  spinal  cord  of  the  lamprey  [1],  and  the  attempts  to 
model  the  experimental  results  using  networks  of  coupled  oscillators  [2]  [3]  [4] 
are  a  good  illustration  of  how  relatively  simple  coupled  oscillator  networks  can 
model  a  variety  of  observed  behaviors  in  an  actual  biological  motor-control 
network.  There  has  also  been  recent  work  in  designing  control  systems  based 
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on  coupled  oscillators  for  both  hexapod  and  biped  robotic  locomotion  [5]  [6] . 


Experiments  on  cats  have  revealed  synchronous  firings  of  neurons  in 
the  cat  visual  cortex  in  response  to  visual  stimulus  [7]  [8].  Moreover,  there 
has  been  work  in  developing  artificial  pattern  recognition  systems  based  on 
networks  of  coupled  oscillators,  in  which  synchronous  oscillations  serve  as  a 
dynamic  binding  mechanism  [9]  [10]  [11].  Moreover,  a  method  of  performing 
systematic  reasoning  based  on  using  synchronous  oscillations  as  a  dynamic 
binding  mechanism  has  been  proposed,  both  for  artificial  information  pro¬ 
cessing  and  as  a  model  sharing  some  features  of  human  reflexive  reasoning 
[12]- 


Besides  biologically  inspired  uses  for  coupled  oscillator  networks,  there 
are  control  problems  for  which  periodic  inputs  have  been  proposed  and  stud¬ 
ied  [13]  [14].  Central  pattern  generators  based  on  networks  of  coupled  oscil¬ 
lators  may  prove  useful  as  control  systems  for  such  problems. 
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1.2  Overview 


The  basic  network  examined  in  this  work  is  given  by 


Vi  =  -yj  +  r(P\xi\)Pi 

(i.i) 

\xj\ 

xj  =  £  j  =  i)  •••,  n 

(1.2) 

jt=i 


where  Xj  and  yj  are  complex  numbers  Vj,  Wjk  are  complex  interconnecting 
weights  with  Wjj  —  0  Vj  and  Wjk  =  w*kj  (i.e.,  the  weight  matrix  is  Hermi- 
tian),  j3  is  a  scalar  parameter,  and  r(-)  :  [0,  oo)  — >■  3?  is  a  memoryless  strictly 
monotone  increasing  nonlinearity  with  r(0)  =  0  and  linv^oo  r(m)  —  1  (later 
to  be  specified  precisely),  r(-)  is  also  assumed  to  be  analytic,  and  the  func¬ 
tion  r{j3\z\)-fa  :  (D  -4  (D  is  a  well-defined  function  which  compresses  the 
magnitude  of  its  complex  argument  while  retaining  its  angle. 

For  the  jth  unit  (or  oscillator),  yj  may  be  thought  of  as  its  state,  and 
Xj  represents  its  input  from  the  rest  of  the  network.  Because  the  states  are 
complex,  each  unit  carries  both  phase  and  amplitude  information,  and  it  is 
the  phase  information  which  is  of  primary  interest.  In  the  coupled  oscillator 
context,  the  relative  phases  of  a  network  of  oscillators  oscillating  at  the  same 
frequency  are  represented. 

The  reason  for  choosing  these  dynamical  equations  is  that  they  repre- 
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sent  a  very  simple  method  of  defining  a  coupled-oscillator  network.  The  input 
to  each  unit  is  the  weighted  sum  of  the  states  of  the  other  units,  where  the 
weights  themselves  are  complex  (and  thus  may  alter  the  phase  as  well  as  the 
amplitude  of  the  signals  they  weight).  The  input  to  each  unit  is  then  passed 
through  a  memoryless  saturating  nonlinearity  whose  gain  is  a  parameter,  and 
the  result  is  then  low-pass-filtered  to  produce  that  unit’s  state. 

In  Chapter  2,  convergence  of  this  network,  in  the  spirit  of  LaSalle’s  In¬ 
variance  Principle,  is  proved,  and  physical  motivation  for  the  proof  is  given. 
Next,  in  Chapter  3,  the  weights  (wjk)  are  allowed  to  be  units  (yi)  themselves, 
and  convergence  of  an  adaptive  control  law  for  such  systems  is  proved.  In 
Chapter  4,  a  6-unit  example  circuit  is  presented  illustrating  how  undesired 
stable  equilibria  can  arise.  Then  two  alternative  approaches  are  then  pre¬ 
sented  which  overcome  the  problem  of  undesired  stable  equilibria  appearing 
in  the  network  dynamics.  Implementation  issues  are  addressed  throughout, 
and  finally,  an  analog  VLSI  approach  for  implementing  such  networks  is 
presented  in  Chapter  5.  In  Chapter  6  actual  VLSI  circuits  which  could  be 
used  to  build  a  coupled  oscillator  network  are  described  in  order  to  permit 
the  tradeoffs  among  power  dissipation,  bandwidth,  and  network  size  to  be 
explored.  Chapter  7  concludes  with  recommendations  for  future  work. 
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Chapter  2 


Fixed- Weight  Network  Analysis 

2.1  Introduction 

The  dynamics  under  consideration  are: 

iij  =  -yj  +  rW\xj\)-^-y  (2-i) 

n 

Xj  =  Y Vkw*jk >  XjE<C,  j  =  l,...,n  (2.2) 

fc= i 

where  Wjk  =  €  (D,  Wjj  =  0  Vjf,  (3  6  is  constant,  and  r(-)  = 

Here,  70  and  Ix  are  the  modified  Bessel  functions  of  the  first  kind  and  of  order 
zero  and  one,  respectively,  and  r(-)  :  [0,  oo)  ->  K,  is  a  memoryless  strictly 
increasing  saturating  nonlinearity  with  r(0)  =  0  and  lirnm^oo  r(m)  =  1.  (The 
choice  of  this  particular  saturating  nonlinearity  will  be  motivated  later). 

The  first  step  in  the  convergence  analysis  is  to  rewrite  the  dynamics  in 
terms  of  the  Xj  variables: 

Xj  =  Y  VkWjk 

k 

=  -Yvkwjk  +  Yr(f3\x*\)j^wjk 
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(2.3) 


=  ~xi  + 

These  dynamics  were  proposed  and  physically  motivated  by  Zemel,  Williams, 
and  Mozer  [15];  however,  their  proof  of  convergence  was  incomplete.  After 
a  review  of  the  physical  motivation,  a  rigorous  proof  of  convergence  of  their 
network  using  LaSalle’s  Invariance  Principle  will  be  presented. 

2.2  Physical  Motivation 

Zemel,  Williams,  and  Mozer  propose  first  a  stochastic  network  of  direc¬ 
tional  units  (complex-valued  random  variables  with  magnitude  one  and  an¬ 
gle  representing  directional  information)  interconnected  by  complex  weights 
which  are  considered  fixed  [15].  The  directional  units  evolve  according  to 
probability  distributions  determined  by  the  other  directional  units  and  in¬ 
terconnecting  weights.  The  stochastic  network  is  then  simplified  using  the 
mean-field  approximation  to  give  a  deterministic  network.  The  purpose  of 
examining  the  stochastic  network  is  that  it  provides  insight  for  the  stability 
analysis  of  the  deterministic  network. 

To  begin  the  stochastic  network  analysis,  consider  a  network  of  direc¬ 
tional  units,  each  represented  by  a  random  variable  Zj  taking  values  on  the 
unit  circle  in  the  complex  plane.  The  directional  units  are  interconnected 
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by  fixed  complex  weights  Wjk  satisfying  Wjj  =  0  and  wkj  =  w*k.  Defining 
Xj  =  zkW*jk  f°  represent  the  interaction  of  unit  j  with  the  rest  of  the  net¬ 
work,  the  mean  value  of  Zj  is  determined  by  the  angle  of  Xj,  and  the  variance 
of  Zj  is  inversely  related  to  the  magnitude  of  Xj  (the  precise  dependence  to 
be  determined  below). 

Next,  a  quadratic  form  representing  “energy”  in  terms  of  unit  states 
and  interconnecting  weights  is  defined: 

£(z)  =  -^zTWz*  =  -\Y.  zjzkwjk,  z  =  (*1,  Zn)T-  (2.4) 

Because  W  is  hermitian  (i.e.,  wkj  =  w*k),  E( z)  is  real- valued.  (This  defini¬ 
tion  of  energy  generalizes  the  Hopfield  energy  function  for  binary  units  [15].) 
Using  Xj  =  Y,k  zkw*jk  and  changing  to  polar  coordinates:  Xj  —  a,jel0lj  and 
Zj  =  e*^  (recall  that  \zj\  is  constrained  to  equal  1),  we  define 

Ej(z)  =  ~  [zjX*  +  (. ZjX*)* 

=  —aj  cos ((f) j  —  otj)  (2.5) 

as  unit  j’s  contribution  to  the  total  energy.  Then  E( z)  =  |  J2j  Ej(z).  (Ob¬ 
serve  that  when  the  angle  of  Xj  is  aligned  with  the  angle  of  Zj,  unit  j  is  in 
a  low-energy  state.)  Introducing  a  “Boltzmann  factor”  (3  (interpreted  as  the 
reciprocal  of  temperature),  and  taking  the  probability  that  the  jth  unit  is  in 
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a  state  Zj  =  e1^  to  be  proportional  to  e  <3E^Z\  we  obtain: 


fz,(Zj)  oze?*  ”*+’-**) 


(2.6) 


where  Zj,  the  state  of  unit  j,  is  a  random  variable  taking  values  on  the 
unit  circle  in  the  complex  plane  and  fzji')  is  a  probability  density  function. 
Appropriate  normalization  gives 


fz3{<t>j ) 


2-Kl0(mj) 


gmj  COs(<j>j  ~<f>j )  ^  =  p  ,  = 


(2.7) 


where  70(-)  is  the  modified  Bessel  function  of  the  first  kind  and  order  zero. 
This  is  known  as  the  Von  Mises,  or  circular  normal,  distribution,  and  it  is  a 
distribution  for  circular  random  variables  having  some  characteristics  similar 
to  the  usual  normal  distribution  for  linear  random  variables  [16].  A  circular 
normal  distribution  is  completely  characterized  by  two  parameters:  a  mean 
direction  0  G  [0,  27t)  and  a  concentration  parameter  m  >  0  which  corresponds 
to  the  reciprocal  of  the  variance  of  a  linear  normal  random  variable. 


Next,  Zemel,  Williams,  and  Mozer  apply  a  mean-field  approximation  to 
come  up  with  a  deterministic  network  model.  In  the  mean-field  approxima¬ 
tion,  the  random  variables  Zj  are  replaced  by  their  means  yj  =<  Zj  >  and 
are  treated  as  independent  (even  though  they  are,  in  fact,  highly  coupled). 
The  mean  <  Zj  >  of  a  Von  Mises  random  variable  is  a  complex  number 
Dj  =  r'Je*'yJ  with  7 j  =  4>j  and  Vj  =  Figure  2.1  shows  rj  as  a  function  of 
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mf  it  is  strictly  monotone  increasing,  passes  through  the  origin,  and  satisfies 


limmj  _>oo  )  1- 


Figure  2.1:  Plot  of  rj  =  (from  [15]). 


The  dynamics  for  the  deterministic  network  are  chosen  to  be 

^  =  ~xi  +  E  y*wjk  (2-8) 

so  that  at  equilibrium,  Xj  =  T,kVkWjk,  in  analogy  with  the  expression  Xj  = 
YjkzkW*jk  f°r  stochastic  network.  Furthermore,  xjj  =  r(p\xj\)-^j,  so  that 
if  Xj  is  viewed  as  determining  the  mean  and  concentration  parameter  of  a 
Von  Mises  distribution  according  to  (2.7),  yj  will  be  the  mean  value  of  the 
distribution.  The  deterministic  dynamics  can  be  expressed  in  terms  of  the 
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(2.9) 


Xj  alone  as: 

^  =  -%  +  £r(/JM  )^w;t 

The  total  energy  for  the  deterministic  network  is  found  by  taking  the 
mean  of  the  total  energy  expression  for  the  stochastic  network  (and  assuming 
the  units  are  independent): 

<  E  >=  -l^yjVkwjk-  (2-10) 

Z  j,k 

Furthermore,  the  “entropy”  for  the  deterministic  network  is  found  by  sum¬ 
ming  the  entropies  of  the  individual  units  of  the  stochastic  network  (again 
using  the  independence  assumption): 

H  =  E  ~Pa/^^  +  log(2nI0Wa,))  .  (2.11) 

With  these  definitions  of  <  E  >  and  H,  a  Lyapunov  function  corresponding 
to  what  Zemel  et.  al.  call  “free  energy,”  F  =<  E  >  —TH,  T  =  can  be 
computed,  and  this  Lyapunov  function  can  be  used  to  prove  convergence  of 
the  deterministic  network  using  LaSalle’s  Invariance  Principle.  The  rigorous 
arguments  behind  this  are  a  main  contribution  of  this  thesis. 

2.3  Proof  of  Convergence 

As  will  now  be  shown,  the  deterministic  dynamics  (2.9)  are  conver- 
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gent:  every  trajectory  converges  to  an  equilibrium  point.  LaSalle’s  Invari¬ 
ance  Principle  is  invoked  to  prove  this,  and  a  Lyapunov  function  based  on 
the  physically  motivated  “free  energy”  is  used. 


The  complex  nonlinear  dynamics  are: 


i]  =  ~Xj  +  2 rWM^wjk>  3  =  !» •••» 


n 


(2.12) 


where  the  Xj  are  complex,  r(m)  =  wkj  =  w*k,  and  Wjj  =  0  Vy .  Letting 
xf  =  Re(xj)  and  Xj  =  Im(xj)  we  can  rewrite  the  dynamics  as 

if  =  -sf  +  SrO? 

ij  =  ~xj  +  'Zf,r(PMXkW3k |a;fcffc Wj~,  i  =  1»  •••>*■  (2-13) 


These  dynamics  give  a  well-defined  vector  field  on  ifi2".  (If  =  0,  we  take 
r((3\xk\)Xk =  0.)  The  equilibrium  points  of 
the  dynamics  are  points  where  xf  =  x[  =  ...  =  if  —  xln  =  0.  For  purposes 
of  the  proof  of  convergence,  the  state  space  is  taken  to  be  #t2n. 


At  any  point  in  the  state  space  5ft2",  except  where  xf  and  x j  are  both 
zero  for  some  j,  we  can  define  a  valid  (nonsingular)  change  of  coordinates 
by  xf  +  ixj  =  a,jeiai,  j  =  1,  ...,n.  In  the  new  coordinates,  the  dynamics 
become: 

aj  =  -aj  +  ^2  r(/3ak)bjk  cos(ak  -  aq  -  9jk)  (2.14) 

k 


li 


(2.15) 


<*  =  -£  r(/3ak)bjk  sm(ak  -  a,  -  9jk) 

ai  k 

where  wfk  +  iw!jk  —  b]kel6^k . 

At  all  points  in  3?2n  where  the  change  of  coordinates  is  valid,  we  define 
the  Lyapunov  function: 

V  =  -  £  r(paj)r({3ak)bjk  cos (ak  -  atj  -  9jk) 

j<k 

~T  £  [~Pajr(Paj)  +  log  (27r/0(/?a,-))]  (2.16) 

3 

where  log  denotes  the  natural  log.  (Keep  in  mind  that  Wjk  =  wL,  wjj  =  0, 
and  (5  =  ^.)  It  turns  out  that  V  can  be  continuously  defined  even  where  the 
change  of  coordinates  is  singular,  because  if  either  ay  or  ak  is  taken  to  be 
zero  in  the  above  formula  for  V,  the  term  r(/3a,j)r(/3ak)bjk  cos (ak  —  aj  —  9jk ) 
will  be  zero  regardless  of  the  value  of  ay  or  ak. 

Calculating  V(v)  =  we  obtain 

f  r  i2 

V(v)  =  -  £  <  Pr'(/3aj)  -aj  +  £  r(Pak)bjk  cos(ak  -  aj  -  9jk) 

j  [  i  k 

+  ^a~  £  r(/3ak)bjk  sm(ak  -  otj  -  9jk)  1  .  (2.17) 

a3  L  k  J  J 

Note  that  r(/3ay)  >  0  Vay  >  0  and  >  0  Vay  >  0.  Also,  r'(/3ay)  — >  |  as 
aj  — >  0,  and  r^a^-  — >  f3r'(0aj)  as  Gy  — ►  0. 


12 


We  now  show  that  the  Lyapunov  function  V(u)  has  bounded  sublevel 
sets,  because  this  will  enable  us  to  exhibit  compact  sets  which  are  positively 
invariant  under  the  dynamics,  as  required  to  apply  LaSalle’s  invariance  prin¬ 
ciple.  Observe  that  the  first  term  of  V,  ~Y!,j<kr(/3aj)r(Pak)bjkCos(ak  — 
otj  —  9jk),  is  bounded  as  a,j  —>  oo  for  any  (or  all)  a3.  A  straightforward 
but  lengthy  calculation  (relegated  to  Appendix  A)  shows  that  fia3r{l3aj)  — 
log  (27r/o(/?Oj))  ->  oo  as  a3  -»  oo.  Then  because  /3a,jr(f3aj)—  log  (2TrI0(f3aj))  > 
0  V<2j  >  0,  and  because  the  terms  /?o;t(oj)  —  log  (27T I0(/3a,j))  appear  summed 
in  V,  it  follows  immediately  that  V  is  radially  unbounded  in  the  a3  (where 
by  definition  V  :  — »  3?  is  radially  unbounded  in  its  argument  a  E  5Rn 

if  V(a)  — >  oo  as  ||a|j  — »  oo  [17]).  Moreover,  since  V  is  continuous  even  at 
points  where  the  change  of  coordinates  is  singular,  we  can  conclude  that  V 
has  bounded  sublevel  sets  when  viewed  as  a  function  of  (xf,x[,  ...yX^x^). 

So  far  we  have  shown  that  there  is  a  Lyapunov  function  V  continuous 
on  all  of  3?2ra,  which  has  bounded  sublevel  sets,  and  which  has  V(u)  <  0 
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provided  v  is  not  an  equilibrium  point  and  provided  v  is  not  a  point  where 
our  change  of  coordinates  is  singular.  What  we  will  now  show  is  that  there 
is  no  loss  of  generality  in  assuming  that  a  trajectory  will  pass  through  points 
where  the  change  of  coordinates  is  singular  only  at  isolated  points  in  time. 

Specifically,  we  will  show  is  that  if  a  trajectory  has  xf(t*)  =  xUt*)  =  0 
for  some  j  €  {1,  ...,n}  and  for  some  t*  then  either  t*  is  an  isolated  point  in 
time  for  which  xf  =  Xj  =  0  or  else  xf  —  Xj  =  0  Vt  >  0.  If  xf  =  Xj  =  0  Vt  > 
0,  then  simply  eliminate  the  xf  and  x j  coordinates  and  consider  the  reduced 
system  of  dimension  3?2n~2.  Repeating  this  test  will  reduce  the  system  to  a 
new  system  with  coordinates  (xf,: if  ...jxf,^)  with  h  <  n.  The  trajectories 
of  the  reduced  system  will  have  xf(t*)  =  Xj(t*)  =  0  only  at  isolated  time 
instants  t*,  and  hence  we  will  have  V  =  0  only  at  equilibrium  points  and  at 
isolated  times  t*.  (It  is  easy  to  verify  that  the  reduced  system  has  exactly 
the  same  form  in  terms  of  dynamics,  equilibria,  and  Lyapunov  function  as 
the  original  system.) 

Analyticity  properties  are  the  key  to  showing  that  the  system  can  be  re¬ 
duced  so  that  x f  =  x j  =  0  for  some  j  only  at  isolated  points  in  time.  That  the 
right-hand-side  of  the  differential  equation  for 

equation  (2.13),  is  analytic  in  the  variables  xf,x{,  ...,xf,Xn  follows  straight¬ 
forwardly  from  basic  properties  of  analyticity  (using  methods  in,  e.g.,  [18]). 
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This  in  turn  implies  that  the  trajectory  (xf  (t),  x[(t), ...,  x%(t),  x^(t))  is  an 
analytic  function  of  t  (the  necessary  results  on  analyticity  and  differential 
equations  can  be  found  in  [19]).  Therefore,  for  any  j  G  {l,...,n}, 
will  be  an  analytic  function  of  t.  Hence  if  xf(t*)  =  Xj(t*)  =  0  then  either  t*  is 
an  isolated  point  in  time  for  which  xf  =  Xj  =  0  or  else  xf  —  x j  =  0  it  >  0. 

We  have  a  well-defined,  C°°  vector  field  (2.13)  on  5R2"  and  a  Lyapunov 
function  V  which  has  bounded  sublevel  sets.  For  simplicity,  think  of  the 
vector  field  abstractly,  as  given  by  v  =  f(v),  with  Lyapunov  function  V(u). 
Also,  V(v)  <0  along  trajectories. 

Fix  c  >  0  and  let 

ttc  =  {ve?R2n\V{v)<c}.  (2.19) 

In  order  to  apply  LaSalle’s  Invariance  Principle,  we  need  to  show  that  Qc  is  a 
compact  positively  invariant  set.  We  have  already  shown  that  the  Lyapunov 
function  has  bounded  sublevel  sets,  and  hence  f2c  is  bounded.  In  fact,  it  can 
be  easily  shown  that  I7C  is  closed  as  well,  and  hence  is  compact.  Positive 
invariance  of  Oc  follows  from  the  fact  that  V(v)  <  0  along  trajectories. 

Theorem  2.1  (LaSalle’s  Invariance  Principle):  Let  0  be  a  compact  set 
and  suppose  the  solution  u(t )  starting  in  f2  stays  in  f2  for  all  t  >  0.  Let 
V  :  f  be  a  continuous  function  such  that  V{u{t))  is  a  monotone  nonin- 


15 


creasing  function  of  t.  Let  E  be  the  set  of  all  points  in  Cl  where  V(u)  exists 
and  equals  zero.  Let  M  be  the  largest  positively  invariant  set  in  E.  Then 
v(t)  approaches  M  as  t  — »  oo. 

Proof :  (See  [17].) 

Theorem  2.2:  The  dynamics  (2.13)  converge  to  an  equilibrium  point. 

Proof:  For  any  initial  condition  u 0,  reduce  the  system  if  necessary 
so  that  we  may  assume  that  the  coordinate  transformation  is  only  singu¬ 
lar  at  isolated  points  in  time.  Also,  choose  c  >  0  to  be  greater  than  or 
equal  to  V(u0).  Then  the  set  Q  in  Theorem  2.1  is  taken  to  be  Qc,  as  de¬ 
fined  earlier,  which  is  a  well-defined  compact  subset  of  SR2"  when  viewed  in 
coordinates.  Because  the  vector  field  is  well  defined,  the 
Lyapunov  function  is  monotone  nonincreasing,  and  the  set  f lc  is  compact, 
it  follows  that  the  trajectory  v{t)  exists  and  stays  in  f lc  Vt  >  0.  Thus,  the 
hypotheses  of  Theorem  2.1  are  satisfied,  enabling  us  to  conclude  that  the 
trajectory  will  converge  to  the  largest  positively  invariant  subset  of  the  set 
of  points  in  Plc  such  that  V(u)  =  0.  But  the  positively  invariant  subset  of 
the  set  of  points  with  V(v)  =  0  are  just  the  equilibrium  points  of  the  system 
which  lie  inside  flc. 

LaSalle’s  Principle  thus  enables  us  to  conclude  convergence  of  any  tra- 
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jectory  to  the  set  of  equilibrium  points  of  the  dynamics,  but  not  to  a  specific 
equilibrium  point.  We  will  now  show,  by  appropriate  choice  of  inner  product, 
that  the  system  follows  gradient  dynamics  except  at  isolated  points  in  time. 
At  points  v  =  (ai, ...,  an,  ai, ...,  an),  define  the  inner  product: 

(vuv2)  =  zfirdiag(/?r,(/3ai), ...,  (3r'((3an),air((3ai), ...,  anr(/?a„))^2.  (2.20) 

Then  (z>,  £)  =  —  duV  •  £  =  (— ' W,  £)  so  that  z>  =  —  W;  i.e.,  the  system 
satisfies  gradient  dynamics  (except  at  isolated  time  instants).  Thus,  we  may 
conclude  that  in  fact  the  dynamics  (2.13)  converge  to  an  equilibium  point. 
Q.E.D. 

Observe  that  we  can  define  two  different  but  related  sets  of  dynamics. 
As  will  be  justified  below,  the  following  dynamics  are  more  readily  imple- 
mentable: 

Vj  =  -yj  +  r(P\xj\)TZl  (2-21) 

\x3 1 
n 

Xj  =  y*wjk  (2-22) 

k= 1 

X  • 

at  equilibrium,  yj  =  r(/3\xj\)-r-^~.  (2.23) 

\xj  I 

On  the  other  hand,  the  following  dynamics  are  more  theoretically  tractable: 


X3  =  ~X3  +  VkWjk 

k 

(2.24) 

(2.25) 
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(2.26) 


n 

at  equilibrium,  Xj  —  ^  Vk'Wjk 

k= 1 

(this  is  the  network  of  Zemel  et.  al.) 

The  convergence  result  just  proved  applies  directly  to  the  second  set  of  dy¬ 
namics.  For  the  first  set  of  dynamics,  the  x3  variables  satisfy  the  same  differ¬ 
ential  equation  as  the  Xj  variables  in  the  second  set  of  dynamics.  However, 
an  additional  argument  (given  in  Appendix  B)  is  needed  to  prove  that  the  yj 
variables  in  the  first  set  of  dynamics  converge  to  r(/3\xj\)j^  at  equilibrium 
even  if  the  weight  matrix  is  singular. 

The  first  set  of  dynamics  is  more  readily  implementable  in  an  analog 
system  (for  instance  an  analog  VLSI  circuit,  but  the  same  argument  would 
apply  for  a  biological  implementation)  because  there  is  a  saturating  summa¬ 
tion  followed  by  an  integration  (or  low-pass  filtering),  with  the  state  variables 
yj  constrained  to  lie  inside  the  unit  circle  (for  initial  conditions  inside  the 
unit  circle).  By  contrast,  the  second  set  of  dynamics  requires  the  storage 
and  update  of  state  variables  taking  values  over  a  possibly  much  larger  sub¬ 
set  of  the  complex  plane,  with  different  Xj  having  possibly  greatly  different 
dynamic  range  requirements.  On  the  other  hand,  the  second  set  of  dynamics 
is  much  easier  to  analyze,  and  the  first  step  in  the  convergence  proof  was 
basically  to  convert  the  first  set  of  dynamics  into  the  second  set  of  dynamics. 
A  recurring  theme  in  this  work  is  that  there  are  two  ways  of  writing  down 
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dynamics  for  what  is  essentially  the  same  network  (related  by  a  change  of 
variables).  One  version  of  the  dynamics  is  more  readily  implemented,  and 
the  other  is  simpler  to  analyze  mathematically. 

2.4  Generalization  to  a  Class  of  Networks 

Although  the  choice  of  r(-)  as  the  ratio  of  bessel  functions  enabled  a 
physical  motivation  to  be  given  for  the  dynamics,  careful  examination  of  the 
proof  of  convergence  reveals  that  as  long  as  r(-)  satisfies  certain  properties, 
the  convergence  result  will  still  hold.  First,  the  function  r(-)  must  be  strictly 
monotone  increasing  with  r(0)  =  0.  Second,  r(-)  must  be  analytic.  Third,  we 
need  to  ensure  that  a  suitable  radially  unbounded  Lyapunov  function  exists: 

V  =  -  r(aj)r(ak)bjk  cos(ak  -  otj  -  Qjk)  +  9(flj)  (2.27) 

j 

where  lim^oo  =  oo,  =  ar'(a),  and  #(0)  -  constant. 

The  ability  to  generalize  the  convergence  proof  to  a  class  of  networks 
in  this  manner  is  important  when  analog  implementations  of  these  networks 
are  considered.  Although  saturating  nonlinearities  can  be  achieved  in  analog 
hardware,  a  saturating  nonlinearity  for  the  magnitude  of  a  complex  number 
which  leaves  the  phase  unaltered  is  more  complicated. 
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Chapter  3 


Adaptive  Control  Law  for  a  Variable- Weight 
Network 

3.1  Introduction 

The  usefulness  of  the  network  presented  in  the  last  section  would  be 
greatly  enhanced  if  the  weights  were  not  constrained  to  be  constant.  An 
adaptive  control  law  for  variable  weights  is  now  described  which  retains  the 
convergence  properties  of  the  fixed-weight  network  -  namely,  the  variable 
weight  network  with  adaptively  controlled  weights  is  shown  to  converge  to 
an  equilibrium  point,  again  using  LaSalle’s  Invariance  Principle.  The  utility 
of  the  rigorous  proof  of  convergence  for  the  fixed-weight  network  is  that  it 
can  be  extended  to  more  general  and  useful  networks,  of  which  the  adaptive 
control  law  described  in  this  chapter  is  an  example. 

3.2  Motivation  for  the  Adaptive  Control  Law 

We  motivate  the  adaptive  control  law  mathematically  by  viewing  the 
dynamics  and  Lyapunov  function  abstractly  and  performing  some  calcula- 
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tions.  Abstractly,  for  the  fixed-weight  case  we  have 


V  = 

f{v,w)  (dynamics) 

(3.1) 

V(u,  w) 

(Lyapunov  function) 

(3.2) 

V(i /,  w)  = 

dV  \ 

=  v,w)>.  (3.3) 


Now  letting  some  of  the  weights  be  feedback  functions,  we  have 

i>  =  f(u,w(u))-\-g(u)  (dynamics)  (3.4) 

V(u,  w(v))  (Lyapunov  function)  (3.5) 

•,  ,  ..  (dV  dV  dw\  ...  ...  ... 

=  (a7  +  a^  *;)  (/("•«’("))  +  »(•')) 

=  -  <f{u,w(v))  +  g(u),f{v,w(u))  +  g(u)>  (3.6) 

provided 

( dV dw\  .  .  . 

[a^d^)n  =  -<9{v)’’,>-  (37) 

In  the  transformed  coordinates,  this  condition  becomes 

=  -[(3r'{(3al)g1(v)---(3r'((3an)gn(v) 

air((3ai)gn+1(u)  ■  ■  ■  anr{f3an)g2n(v)]-  (3.8) 


Suppose  unit  j  interconnects  units  l\  and  Z2>  and  all  other  weights  are 
constants.  Furthermore,  suppose  bilti2  depends  only  on  aj  and  Oilti2  depends 
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only  on  oij.  Then 


dV  dw 
dw  dv 


db 

0---0  -r(f3ah)r(pal2)-^cos(ah-ah-dhh)  0---0 

d9 

0---0  -  r(/to|1)r(/)nll)iIlI,  sin(«,,  -  -  0uh)  0  -  •  -  0] 

(3.9) 


where  the  first  nonzero  term  is  in  position  j  of  the  row  vector,  and  the  second 
nonzero  term  is  in  position  j  +  n. 


If  we  choose  blul2(aj)  =  dhhr((3aj)  and  Oiul2(aj ) 
dh,hPr'(Paj)  and  so 


a.j,  then 


dbh  ,h.  __ 

daj 


dV  dw 
dw  dv 


Pr\Pai){diuhr(<Pah)r(Pah)  cos(ah  -  ah  -  a,))  0  •  •  •  0 


0  •  ••Oajr(paj)  —dhhr(pah)r(pah)  sin(a/2  -  ah  -  aj) 


a 


(3.10) 


where  again  the  nonzero  terms  are  in  positions  j  and  j  +  n  of  the  row  vector. 


Thus,  letting 

gj(v)  =  diuhr{(3ah)r(f3ai2)  cos{ah  -  ah  -  aj) 

9j+n{y)  =  —diui2r(^ah)r(f3ai2)sin(ai2-ah-a:l)  (3.11) 

Oj 

gk(v)  =  0,  k^j,j  +  n 


we  then  have 


(dV  dw 


(3.12) 
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Generalizing  to  an  arbitrary  number  of  variable  weights,  we  have 


1  :  bhheid,i’l2  =  dhhr(paj)eia> 

0  :  otherwise 


(3.13) 


dj  =  -a,-  +  r(Pak)bjk  cos(at  -  a,  -  6jt) 

k 

+  phjL2dh*2r(Pah)r(Pah)  cos(ah  -  ah  -  aj)  (3.14) 

h,h 

dj  =  —{J2  r((3ak)bjk  sin(ak  -  etj  -  6jk) 
ai  k 

+  Z  l3lul2dhhr(Pah)r(Pah)  sinK  -  ah  ~  ®j))  (3.15) 

h,h 


which  (provided  the  change  of  coordinates  is  nonsingular)  is  equivalent  to 


1  :  wlul2=dluhr((3\xj\)^ , 

0  :  otherwise 


(3.16) 


=  -^j  +  Y/r{/3\xk\)-~w*k 

+  E  \)r(P\xh  I)  ,Xh  |  u  1 1  (3-17) 

h,h  ■  \xh\\xh\ 

where  the  conditions  biuhel0‘ i>‘2  =  dlui2r(/3dj)ezai  and  wtui2  =  diui2r(/3\xj\)j^ 
simply  indicate  that  unit  j  is  serving  as  the  interconnecting  weight  between 
units  l\  and  /2)  with  the  sense  of  the  connection  (recall  that  wkj  =  w*k)  taken 
into  account. 


3.3  Proof  of  Convergence  for  the  Adaptive  Control 
Law 


To  prove  convergence  of  the  adaptive  control  law,  we  start  with  the  dy- 
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namical  equations  just  given,  (3.13)-(3.17),  along  with  the  Lyapunov  function 
V  =  -  Y  r(Paj)r(/3a,k)bjk  cos (ak  -  aj  -  9jk) 

j<k 

~T  Y  [-Pair(Pai)  +  loS  (2tt/0 {Paj))]  (3.18) 

j 

where  now  we  may  have  bjk  =  djkr(Pai)  and  0jk  =  ±cq  for  various  j,  k,  l. 
The  new  calculation  of  V  gives: 

V(y)  =  -  Y  { Pr'(Paj)  ~aj  +  Y  r(Pak)bjk  cos(ak  -  otj  -  9jk) 

j  L  k 

1  2 

+  Y  Ih,hdhhr{Pah)r{Pah)  cos(ah  -  ah  -  aj) 

hyh 

t  ( 3d  ^  r 

+— — —  Y  r(Pak)bjk  sin(o:fc  -  otj  -  djk ) 

ai  1  k 

121 

+  Y  Ih,hdh,hr(Pah)r(f3ah)  sin(o:/2  -  ah  -  aj)  >.  (3.19) 

hh  J  J 

From  this  point  on,  the  proof  of  convergence  is  basically  the  same  as  the 
convergence  proof  for  the  fixed-weight  case.  The  reason  for  labeling  the 
feedback  law  an  adaptive  control  law  is  that  the  original  dynamics  were  linear 
in  certain  parameters  (the  weights),  which  are  now  adapted  according  to  a 
feedback  law  which  guarantees  convergence.  In  this  way,  coupled  oscillator 
networks  can  be  designed  to  adapt  in  the  presence  of,  for  example,  fixed  but 
(a  priori)  unknown  weight  values,  as  the  6-unit  feedback  network  presented 
in  the  next  chapter  will  illustrate. 

The  implementable  version  of  these  dynamics  is  derived  in  Appendix 


24 


C.  The  implementable  dynamics  are 


Vi  =  ~Vj  +  r(P\xi\) 


b,- 


XI 


Xj  =  E  Vkwjk  +  WO  -  E  |)  |  , 

<1,12  I  1  I 


(3.20) 

(3.21) 


fc=i 


"0  =  -W(j  +  E  {iLAm  +  I^djJI}r(PM)rWxh\)^^ 


h,h 


x  ■ 

at  equilibrium,  yj  =  r(f3\xj\)-r-^7- 


Xi 


(3.22) 

(3.23) 


The  theoretically  tractable  dynamics  are 


Xj  =  -Xj  +  E  y*wjk  +  E  ^hh^uhVhVt 

k  l\  ,^2 

Vj  =  r(P\xj\) 


Xj 

\x< 


at  equilibrium,  Xj  =  E  VkW*jk  +  E  ^hM^hVhVh- 

k= 1  h,h 


(3.24) 

(3.25) 

(3.26) 


As  was  the  case  for  the  fixed-weight  network,  the  implementable  and 
theoretically  tractable  dynamics  share  the  same  convergence  properties.  The 
implementable  dynamics  still  consist  of  a  saturating  summation  followed  by 
a  low-pass  filtering,  except  that  now  some  of  the  terms  which  are  summed  by 
the  saturating  summer  are  low-pass  filtered  first.  Furthermore,  the  function 
r(-)  may  be  generalized  in  the  same  way  as  for  the  fixed-weight  network, 
which  in  turn  gives  a  convergent  adaptive  control  law  for  a  class  of  networks 
of  nonlinear  coupled  oscillators. 
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Chapter  4 


Eliminating  Undesired  Stable  Equilibria 

4.1  Introduction 


To  illustrate  how  coupled  oscillator  networks  may  be  employed,  and 
to  motivate  why  it  is  necessary  to  devise  a  method  of  eliminating  undesired 
stable  equilibria,  we  now  present  and  begin  the  analysis  of  the  simple  6-unit 
feedback  circuit  shown  below: 


Figure  4.1:  6-unit  feedback  circuit. 
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0  represents  an  unkown  phase  shift  between  units  1  and  6,  and  as  will 
be  shown,  the  circuit  acts  to  align  the  phase  of  unit  6  with  ir  +  ip,  a  reference 
phase.  Unit  5  interconnects  units  1  and  2,  and  obeys  the  adaptive  control 
law. 


Since  we  know  the  network  will  converge  to  an  equilibrium  point,  the 
first  step  is  to  find  all  of  the  equilibrium  points  and  determine  which  are 
stable.  The  equilibrium  equations  in  transformed  coordinates  are 

di  =  —  eq  +  r(/3a2)r(/?as)  cos(a2  —  ol\  —  as)  +  r(/?a4)  cos(a4  —  a4  —  7r) 
+r(/?a6)  cos(a6  -  a4  +  0)  =  0 

a2  =  -a2  +  r(/3ai)r(/3a5 )  cos(a4  -  a2  +  as)  +  r(/?°3)  cos(a3  -  a2)  =  0 

a3  =  -a3  +  r(/?a2)  cos(a2  -  a3)  +  r(/?a4)  cos(a4  -  a3  +  0)  =  0 

a4  =  — a4  +  r(/?a4)  cos(ai  -  a4  —  7r)  +  r(^a3)  cos(a3  —  a4  — -0)  =  0 

05  =  -05  +  r(/3a6)  cos(a6  -  a5)  +  r(/?a2)r(/?ai)  cos(a2  -  a4  -  a5)  =  0 

a6  =  — a6  +  r(/3ai)  cos(ai  —  a6  —  9)  +  r(/?a5)  cos(c>!5  —  a6)  =  0 

di  =  —  [r(/?a2)r(/?a5)  sin(a2  -  a4  -  a5)  +  r(/?a4)  sin(a4  -  a4  -  7r) 
a4 

+r(/?a6)  sin(a6  -  a4  -f  0)]  =  0 

a2  =  —  [r(/?ai)r(/?a5)  sin(ai  -  a2  +  a5)  +  r(/?a3)  sin(a3  -  a2)]  =  0 
u2 

a3  =  —  [r(/?a2)  sin(a2  -  a3)  +  r(/?a4)  sin(a4  -  a3  +  ip)\  =  0 

®3 

d4  =  —  [r(/3a4)  sin(o!i  —  a4  —  7r)  +  r(/?a3)  sin(a3  —  a4  —  -0)]  =  0 
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«5  =  —  [r(/?a6)  sin(o6  -  a5)  +  r(/3a2)r(/3oi)  sin(a:2  -  aq  -  <25)]  =  0 
a  5 

«6  =  — ■  [r(^oi)  sin(afi  —  a6  —  9)  +  r(pas)  sin(a:5  —  a:6)]  =  0. 
a6 

(4.1) 

The  equilibrium  phases  we  would  like  to  show  to  be  the  only  stable  phases 
satisfy 

O'  2  —  Q!i  —  Q!  5  =  0 
O4  —  Qq  —  7T  =  0 

a6  —  cti+9  =  0  (4.2) 

013  —  0:2  =  0 

04  —  0:3  +  -0  =  0 

a  6-0:5  =  0. 

The  solution  to  this  system  of  linear  equations  is 


Oi  =  9  +  *p  +  7T 
a2  =  9  +  2*p 

o3  =  9  +  2*1; 

oq  =  9  +  *p 

0:5  =  Ip  +  7T 

a6  =  tp  +  TT. 


(4.3) 
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However,  although  we  would  like  to  proceed  with  the  equilibrium  point  iden¬ 
tification  and  stability  analysis  in  the  transformed  coordinates,  there  are  two 
problems:  first,  there  may  be  equilibria  with  one  or  more  of  the  a,  equal 
to  zero;  and  second,  identifying  all  the  equilibria  (which  means  solving  the 
system  of  nonlinear  algebraic  equations)  is  not  a  straightforward  problem. 
Thus  there  may  be  undesirable  stable  equilibria  in  addition  to  the  desired 
stable  equilibrium  point. 

There  are  two  distinct  approaches  one  could  take  to  eliminate  unde¬ 
sired  stable  equilibria.  The  first  approach  is  to  use  extra  directional  units 
configured  to  prevent  the  original  network  of  units  from  having  equilibrium 
points  with  one  or  more  of  the  aj  being  zero.  We  will  show  that  by  coupling 
one  extra  unit  to  each  unit  in  the  original  network  we  can  prevent  any  of  the 
cij  from  being  zero  at  a  stable  equilibrium  point.  The  second  approach  is  to 
modify  the  network  dynamics  so  that  the  a,j  are  constrained  from  being  zero 
at  equilibrium  without  the  need  for  extra  units.  We  will  show  that  by  simply 
assigning  a  value  to  the  magnitude  of  a  unit  and  eliminating  the  equation 
for  that  unit’s  magnitude  from  the  dynamics,  the  same  form  of  Lyapunov 
function  still  works  for  proving  convergence.  Roughly  speaking,  the  first  ap¬ 
proach  (adding  units)  is  more  implementable  for  an  analog  system,  while  the 
second  approach  (modifying  the  dynamical  equations)  is  simpler  to  imple- 
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ment  in  a  digital  system.  Again,  the  arguments  used  in  the  rigorous  proof  of 
convergence  for  the  fixed- weight  network  will  be  instrumental  in  solving  this 
technical  problem,  which  must  be  overcome  if  such  networks  are  to  be  useful 
in  practice. 

4.2  Digital  Implementation  Approach 

The  new  dynamical  equations,  with  the  a,j(t)  =  hj  constant  functions 
of  time,  are 

a,  =  ~  (  E  r(pak)bjk  sin(ak  -  a,-  -  9jk)  (4.4) 

aj  k 

+  E  sinK  “  ah  ~  aj))- 

h,h 


The  Lyapunov  function  has  the  usual  form: 


V  =  -  53  r(fiaj)r(pak)bjk  co s(ak  -  a j  -  0jk) 

j<k 

-T  J2  [-PtyriPaj)  +  log  (2nl0{/3aj))) .  (4.5) 

j 


Because  the  term  —  T  Yj  [— /?ajr(/?Oj)  +  log  (27r/o(/?fij))]  is  constant,  we 
can  just  as  well  use  the  Lyapunov  function 


V  =  -  r(/3aj)r((3ak)bjk  cos(ak  -  aj  -  9jk).  (4.6) 

j<k 
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The  convergence  proof  for  this  network  is  simpler  than  the  general 
adaptive  control  law  proof,  because  the  system  is  evolving  on  the  n-torus, 
T 11 ,  rather  than  on  d".  is  now  simply  Tn,  which  is  a  compact  manifold. 
V(u)  for  u  =  (ax, ... ,an )  is  computed  to  be 

V(v)  =  -  2  53  riPak)bjk  sin(o!^  -  aj  -  9jk) 

j  K  ai  k 

l2'! 

+  Y  l3h,hdh ,hr(Pah)r(Pah)  sin(o:/2  -  ah  -  aj)  \.  (4.7) 

hh  J 

Thus,  V  <  0  and  V  =  0  only  at  equilibrium  points  of  the  dynamics. 
Applying  LaSalle’s  Invariance  Principle,  we  may  conclude  convergence  to  the 
set  of  equilibrium  points.  Furthermore,  the  dynamics  are  gradient  dynamics, 
as  we  may  define  the  inner  product 

v-i )  =  f/i’diag(air(/?oi), ...,  anr(pan))u2.  (4.8) 

As  before,  (v,()  =  —duV  •  (  —  (-VV,  ()  so  that  z>  =  —  VV.  Thus,  we  may 
conclude  that  the  system  actually  converges  to  an  equilibrium  point. 

Observe  that  the  dynamics  (4.4)  are  readily  implementable  in  digital 
hardware,  because  the  aj  and  aj  remain  bounded  (with  a  bound  which  is 
easily  computed).  Furthermore,  r(-)  only  appears  as  its  values  at  fldi, ...,  jdan. 

It  may  not  always  be  desirable  to  prevent  every  unit  in  the  network 
from  being  able  to  approach  zero.  The  above  proof  can  easily  be  extended  to 
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accomodate  having  certain  units’  magnitudes  fixed,  but  allowing  other  units’ 
magnitudes  to  vary  according  to  the  adaptive  control  law. 

4.3  Analog  Implementation  Approach 

The  dynamics  (4.4),  although  easily  implemented  in  digital  hardware, 
are  poorly  suited  to  analog  implementation  due  to  the  need  to  calculate  the 
sine  of  various  quantities.  An  alternative  approach  to  modifying  the  dy¬ 
namical  equations  (3.20)  -  (3.26)  in  order  to  eliminate  the  undesired  stable 
equilibria  is  to  instead  modify  the  network  topology.  As  will  now  be  shown, 
by  coupling  an  extra  unit  to  each  unit  in  the  original  network,  stable  equi¬ 
libria  with  one  or  more  of  the  aj  equal  to  0  can  be  eliminated. 

To  illustrate  the  idea  behind  the  approach  of  eliminating  undesired  sta¬ 
ble  equilibria  by  coupling  an  extra  unit  to  each  unit  in  the  network,  consider 
the  following  two-unit  network: 


7 


Figure  4.2:  2-unit  network. 
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The  complex  dynamics  for  this  system  are 


%2 


xx  =  -Xi+r(P\x-,\)- — r7 

F2I 


^2 


Xx 


-rc2  +  r(/?|:r1|)— 7. 


(4.9) 

(4.10) 


We  can  show  that  this  system  is  unstable  at  the  origin  by  linearization.  The 
Jacobian  (found  by  expanding  I0  and  lx  about  zero)  is 

(4.11) 

Thus,  the  origin  will  be  unstable  provided  /?7  >  2. 


_1  tL 
„ 1  2 
£l 

L  2  l 


Now  we  consider  unit  1  to  be  connected  to  other  units,  and  in  addition 
unit  1  is  connected  to  an  auxiliary  unit,  unit  1’,  as  shown: 


Figure  4.3:  Unit  connected  to  auxiliary  unit  and  to  rest  of  network. 
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The  equilibrium  equations  for  units  1  and  V  are 


~xi+  ^r(P\xk\)^™lk  +  r(PM)^1  =  0  (4.12) 

—x\'  +  r(/3|xi|)p^j7  =  0.  (4.13) 

If  £*#i'  r(0M)|jjJiU>ifc  ^  0  then  clearly  the  equilibrium  equations  cannot  be 
satisfied  if  either  xx  —  0  or  xy  =  0.  On  the  other  hand,  if  Y^k^v  r(/3\xk\)j^w*k 
0  at  equilibrium  and  we  let  Xi  —  Xy  =  0,  then  we  have  an  unstable  linear  sys¬ 
tem  subject  to  a  nonlinear  perturbation  which  converges  to  zero.  Therefore, 
we  may  conclude  that  the  equilibrium  point  x\  =  xy  =  0  is  still  unstable. 

Since  at  no  stable  equilibrium  point  do  we  have  X\  =  0  or  xy  =  0, 
we  can  perform  the  rest  of  the  analysis  in  transformed  coordinates.  The 
dynamics  for  ay  is 

ay  =  — jr(Pai)  sin(ai  —  ay).  (4.14) 

ay 

At  equilibrium,  this  implies 

sin(ai  -  ay)  =  0.  (4.15) 

We  would  like  to  show  that  the  only  stable  solution  is  aq  —  ay  =  0.  However, 
this  will  be  a  consequence  of  the  following  result,  which  can  also  be  extended 
to  show  that  for  the  system  of  figure  4.1  with  an  additional  unit  coupled 
to  each  unit  shown,  the  only  stable  equilibrium  point  is  the  desired  stable 
equilibrium  point: 
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Theorem  4-1:  (Result  on  stability  of  equilibrium  points  for  fixed- weight 
networks)  If  the  following  conditions  are  satisfied: 

(1)  The  network  consists  of  interconnected  units  and  is  such  that  no  stable 
equilibria  exist  with  one  or  more  aj  =  0. 

(2)  The  network  is  connected;  i.e.,  the  network  cannot  be  separated  into 
two  or  more  subnetworks  whose  convergence  could  be  analyzed  sepa¬ 
rately. 

(3)  All  equilibrium  points  satisfy  sin(o:jt  —  aj  —  9jk)  =  0  Vj,  k  such  that 
bjk  ^  0. 

(4)  The  interconnecting  weights  are  fixed. 

Then  the  unique  stable  equilibrium  point  satisfies  cos(«fe  —  aj  —  6jk)  =  1  Vj,  k 
such  that  bjk  ^  0. 

Proof:  Suppose  for  an  n-unit  fixed-weight  network  for  which  no  stable 
equilibria  exist  with  one  or  more  aj  =  0,  3p,  1  <  p  <  n,  such  that  at 
equilibrium 

cos (ak  —  aj  -  6jk)  —  -1  for  k  =  p  +  1, ...,  n  (4.16) 

bjk  #  0 

l  =p+  1,  ...,n 

cos (ak-ai-8ik)  =  1  for  <  k  =  p  +  1, ...,  n  (4.17) 

.  bik  7^  0 
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cos(am  —  aj  —  9jm )  =  1  for  m  =  1, 2,  ...,p  (4-18) 

,  ~f~  0- 

(To  see  that  this  covers  all  possible  cases  for  undesired  stable  equilibria,  en¬ 
vision  starting  out  with  each  unit  satisfying  the  desired  stable  equilibrium 
configuration,  and  then  changing  the  phase  of  certain  units  by  7r.  This  in¬ 
volves  no  loss  of  generality  because  for  fixed-weight  networks  the  equilibria 
are  classified  by  phase  differences  rather  than  absolute  phases.  In  other 
words,  if  a  given  set  of  phases  represents  an  equilibrium  point,  adding  a 
constant  to  each  phase  gives  a  different  set  of  phases,  but  the  new  set  also 
represents  an  equilibrium  point  with  the  same  stability  properties.) 


Let 


aj  =  oij  +  SjAa 


where 


1  3  =  1  ,-,P 
0  j  =p+l,...,n 


and  0  <  |Aq<|  <<  1.  Letting  AV  =  V{y)  —  V(u),  we  have 


(4.19) 

(4.20) 


AV  =  Y,  r{P<*j)r(Pak)bjk[-  cos{(ak  -  atj  -  9jk)  +  ( Sk  -  Sj)  Aa) 

j<k 

+  cos(afc  -  aj  -  9jk)}  (4.21) 

=  ^2  r(Paj)r(f3ak)bjk  cos (ak  -  olj  -  9jk)  [1  -  cos{{Sk  -  Sj)  Aa)} . 

j<k 

(4.22) 
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Assumption  (2)  implies  that  the  summation  above  must  have  at  least  one 
nonzero  term.  Furthermore,  [1  -  cos((Sk  -  S^Aa)]  >  0  for  j  <  p  and  k  >  p, 
which  is  precisely  when  cos(afc  -  a,  -  9jk)  =  -1.  Otherwise,  [1  -  cos ((Sk  - 
5j)Ao;)]  =  0.  Thus,  AV  <  0,  which  means  there  is  a  descent  direction 
at  any  equilibrium  point  which  we  desire  to  be  an  unstable  equilibrium 
point.  Moreover,  the  dynamics  in  a  neighborhood  of  one  of  these  equilib¬ 
rium  points  are  gradient  dynamics  (assumption  (1)  guarantees  the  metric  is 
well-defined  in  a  neighborhood  of  the  equilibria).  Thus,  all  equilibria  not 
satisfying  cos(afc  -  aj  -  6jk)  =  1  Vj,  k  such  that  bjk  ^  0  must  be  unstable. 
Q.E.D. 

Theorem  4-2:  The  network  of  figure  4.1,  with  an  auxiliary  unit  con¬ 
nected  to  each  unit  shown  to  preclude  stable  equilibria  with  one  or  more 
cij  =  0,  has  the  unique  stable  equilibrium  point  given  by  (4.3). 

Proof:  The  proof  of  theorem  4.1  must  be  generalized  slightly  for  the 
case  at  hand,  in  which  one  of  the  weights  is  nonconstant  and  is  adaptively 
controlled.  (4.16)  -  (4.18)  still  apply,  exluding  the  case  when  Wjk  is  the  adap¬ 
tively  controlled  weight  in  (4.16),  when  wik  is  the  adaptively  controlled  weight 
in  (4.17),  and  when  Wjm  is  the  adaptively  controlled  weight  in  (4.18).  The 
justification  for  using  (the  modified)  (4.16)  -  (4.18)  can  no  longer  be  based  on 
the  fact  that  for  fixed-weight  networks  only  the  relative  phases  matter,  be- 
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cause  the  absolute  phases  are  important  in  the  adaptively-controlled-weight 
case.  Here  we  can  use  the  fact  that  sin(^)  =  0  implies  cos ((f))  =  ±1,  and  view 
(4.16)  -  (4.18)  as  simply  an  enumeration  of  which  cosine  terms  are  1  and 
which  are  -1.  (The  relatively  simple  interconnection  of  units  in  this  network 
makes  it  easy  to  see  that  this  is  possible.) 

We  define  a  as  in  (4.19).  However,  Sj  must  now  be  chosen  based  on 
whether  cos (ak  —  aj—ai)  —  1  or  —1,  where  unit  l  is  the  adaptively  controlled 
interconnecting  weight  for  units  j  and  k.  The  equation  for  AH  is  now 

AH  =  Y,  r(paj)r(pak)bjk  cos(ak  -  ctj  -  0jk)[\  -  cos {{Sk  -  Sj  -  Sjk)  Ac*)] 

j<k 

(4.23) 

where  Sjk  =  Si  if  unit  l  is  the  adaptively  controlled  weight  and  sjk  =  0 
otherwise. 


If  cos  (ak  —  aj—ai)  =  —1,  then  Sj  may  be  defined  as  in  (4.20).  However, 
if  cos(ak  —  aj  —  ai)  =  1,  then  Sj  must  be  chosen  to  make  Sk  —  Sj  —  Si  —  0. 
Sj  must  also  satisfy 

Ci  j  =  1 

C2  j  =p  +  l,...,n 

where  Ci  and  c2  are  constants  with  ci  c2.  (Regardless  of  the  values  of  cx 
and  c2,  the  fixed-weight  terms  of  AH  will  be  nonpositive.)  Given  whether  j, 
k,  and  l  are  greater  than  or  less  than  p,  it  is  clearly  possible  to  find  ci  and 
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C2  so  that  Sk  —  Sj  —  Si  —  0  is  satisfied. 


(There  is  one  final  special  case  to  take  into  account:  the  p  =  0  case 
with  cos(a:jfc  —  a.j  —  on)  =  —  1.  An  appropriate  function  Sj  giving  AV  <  0  is 
readily  obtainable  in  this  case  as  well.) 

Hence,  as  in  the  proof  of  Theorem  4.1,  AV  <  0,  which  implies  that  all 
equilibria  not  satisfying  cos (ak  —  aj  -  6jk)  =  1  Vj,  k  such  that  bjk  ±  0  must 
be  unstable.  Therefore,  the  unique  stable  equilibrium  phases  for  the  system 
of  figure  4.1  are  given  by  (4.3).  Q.E.D. 

Thus,  as  claimed,  the  circuit  of  figure  4.1  acts  to  align  the  phase  of 
unit  6  with  7r  +  ■)/>,  a  reference  phase,  regardless  of  the  unknown  phase  shift 
6  between  units  1  and  6. 
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Chapter  5 


Analog  VLSI  Implementation 

5.1  Introduction 

There  are  several  reasons  to  consider  attempting  to  implement  these 
networks  of  coupled  oscillators  in  analog  VLSI  hardware.  Analog  processing 
at  the  output  of  receivers  for  communications  or  radar,  or  at  the  output  of 
smart  sensors,  can  improve  system  performance  if  the  analog  processing  can 
be  made  to  work  well  enough.  The  principle  limitation  for  analog  continuous¬ 
time  silicon  CMOS  circuits  is  generally  threshold  voltage  mismatch  between 
nominally  identical  transistors.  Such  threshold  voltage  mismatches,  or  off¬ 
sets,  will  affect  how  these  coupled  oscillator  networks  perform.  However, 
the  advantage  the  network  of  coupled  oscillators  possesses  is  that  by  keep¬ 
ing  the  unit  magnitudes  fairly  large  (at  least  around  stable  equilibria),  the 
signal  levels  within  the  coupled  oscillator  network  can  be  maintained  at  a 
high  enough  level  that  the  offsets  only  introduce  a  small  relative  error.  Fur¬ 
thermore,  expressing  the  dynamics  in  the  implementable  form  (2.21)-(2.23) 
or  (3.20)-(3.23)  aids  in  analyzing  offset  effects,  in  contrast  to  the  theoreti¬ 
cally  tractable  form  (2.24)-(2.26)  or  (3.24)-(3.26)  which  fails  to  indicate  the 


40 


dynamic  range  requirements  of  the  Xj. 

Analog  VLSI  is  the  most  realistic  option  for  implementing  these  net¬ 
works  in  an  analog  form,  because  the  number  of  multipliers  required  is  fairly 
large  even  for  networks  consisting  of  a  modest  number  of  units.  Only  sili¬ 
con  CMOS  implementation  will  be  considered  in  this  work,  because  silicon 
CMOS  is  the  most  widely  used  and  available  technology  for  prototyping  ana¬ 
log  VLSI  circuits.  Furthermore,  to  make  the  discussion  more  concrete,  the 
MOSIS  2  n m  technology  will  be  considered  [20]. 

Processing  nonuniformities  cause  slight  variations  in  threshold  voltage, 
even  for  nominally  identical  transistors  located  in  close  proximity.  Based  on 
the  author’s  experience  in  testing  analog  CMOS  circuits  at  the  U.S.  Naval 
Research  Laboratory’s  Microelectronic  Device  Physics  Section  as  part  of  the 
analog  VLSI  adaptive  filter  project  [21]  [22]  [23]  [24],  it  was  generally  found 
that  nominally  identical  transistors  within  a  circuit  (i.e.,  transistors  not  wire- 
bonded  to  package  leads)  and  located  in  close  proximity  had  threshold  mis¬ 
matches  in  the  2mV  to  3mV  range.  However,  transistors  with  gates  bonded 
to  package  leads  regularly  exhibited  threshold  mismatches  as  high  as  200mV. 
One  possible  conclusion  from  this  observation  is  that  during  the  wire-bonding 
process,  charge  gets  trapped  in  the  gate  oxide,  causing  a  threshold  voltage 
shift.  For  design  work,  the  threshold  mismatch  between  nearby  identical 
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transistors  was  taken  to  be  lOmV,  as  recommended  in  [25],  since  nearly  all 
threshold  offsets  would  be  expected  to  be  below  that  value,  and  care  was 
taken  with  respect  to  how  transistor  gates  were  brought  to  bonding  pads. 

The  purpose  of  this  chapter  is  to  sketch  out  an  approach  for  imple¬ 
menting  networks  of  coupled  oscillators  in  analog  VLSI  in  order  to  illuminate 
important  practical  issues.  Two  methods  of  producing  the  complex  saturat¬ 
ing  nonlinearity  are  presented,  and  the  extra  dynamics  they  introduce  are 
discussed. 

5.2  Complex  Saturating  Nonlinearities 

5.2.1  Introduction 

The  first  challenge  in  devising  an  analog  implementation  of  either  the 
fixed-weight  dynamics  (2.21)-(2.23)  or  the  adaptively  controlled  dynamics 
(3.20)-(3.22)  lies  in  how  to  effect  the  complex  saturating  nonlinearity  Xj  (->■ 
r(fi\xj\)j^7\-  Tw0  quite  different  approaches  will  be  presented:  a  quadra¬ 
ture  oscillator  approach  and  an  automatic  gain  control  approach.  In  both 
cases,  it  will  be  shown  that  the  complex  saturating  nonlinearity  introduces 
a  high-frequency  pole  into  the  dynamics,  and  that  this  pole  will  limit  the 
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maximum  corner  frequency  of  the  network  low-pass  filters  (to  a  corner  fre¬ 
quency  well  below  the  corner  of  the  high-frequency  pole).  The  situation  is 
similar  to  voltage  op-amp  design,  where  the  ultimate  bandwidth  limitation 
of  a  normal  voltage  op-amp  is  due  to  the  high-frequency  non-dominant  pole 
rather  than  the  low-frequency  dominant  pole  [26] .  The  quadrature  oscillator 
approach  has  the  advantage  of  being  conceptually  simple,  but  the  automatic 
gain  control  approach  will  be  shown  to  require  fewer  components. 

5.2.2  Quadrature  Oscillator  Approach 

The  quadrature  oscillator  approach  for  producing  the  complex  saturat¬ 
ing  nonlinearity  is  shown  in  figure  5.1.  A  reference  oscillator  generates  I  (in- 
phase)  and  Q  (quadrature)  signals,  which  are  90  degrees  out  of  phase.  These 
high-frequency  signals  multiply  the  real  and  imaginary  parts  of  each  Vkwjk 
term,  and  summing  the  high-frequency  signals  produces  a  high-frequency  sig¬ 
nal  whose  phase  contains  the  directional  information  present  in  the  real  and 
imaginary  parts  of  ykWjk-  For  each  unit  j,  the  high-frequency  signals  ykW*jk 
are  summed  (to  produce  Xj)  and  passed  through  a  real  saturating  nonlin¬ 
earity.  At  the  output  of  the  real  saturating  nonlinearity,  the  high-frequency 
sum  signal  for  each  unit  (which  is  now  a  distorted  sine  wave)  is  multiplied 
by  I  and  Q  again,  and  low-pass- filtered  to  give  the  real  and  imaginary  parts 
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of  r((3xj) 


Figure  5.1:  Quadrature  oscillator  approach. 


This  approach  has  the  advantage  of  being  conceptually  simple,  but 
there  are  two  drawbacks.  The  first  drawback  is  that  the  VkWjk  signals  must 
be  mixed  up  to  the  carrier  frequency  before  they  can  be  summed  over  k  for 
each  j.  It  is  not  possible  to  first  sum  the  over  k  to  produce  the  Xj 

and  then  mix  the  Xj  up  to  the  carrier  frequency  to  be  passed  through  the 
saturating  nonlinearity,  because  the  saturation  operation  must  take  place  at 
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the  point  where  the  ykWjk  are  summed.  (Otherwise,  the  real  and  imaginary 
parts  of  Vkwjk  saturate  individually  and  the  phase  information  in  Xj 
will  be  lost.)  As  will  be  shown  when  actual  circuits  are  considered,  what 
are  summed  are  currents,  and  the  sum  current  can  be  converted  to  a  voltage 
using  a  nonlinear  resistance  to  produce  the  saturating  nonlinearity. 

The  second  and  more  subtle  drawback  is  the  need  to  properly  handle 
the  high-frequency  signals  as  the  UkWjk  are  being  summed  and  saturated  to 
produce  the  r(/3xj) ^y.  Any  phase  shift  of  the  high-frequency  signal  as  it 
is  being  passed  through  the  saturating  nonlinearity  will  result  in  a  phase 
error  once  the  r(f3xj)  ^  are  demodulated.  If  too  great,  the  high-frequency 
phase  shift  can  cause  the  network  of  directional  units  to  become  unstable  (or 
more  specifically,  to  oscillate  with  a  stable  limit  cycle  solution).  Therefore, 
there  is  a  limit  to  how  high  the  I-Q  frequency  can  be  for  the  network  to  still 
converge  to  a  stable  equilibrium  point  of  the  dynamics.  Furthermore,  since 
the  high-frequency  content  of  the  r(/3xj)  ^  signal  must  be  stripped  so  that 
it  will  not  appear  on  yj,  the  network  low-pass-filter  corner  frequency  must 
be  significantly  lower  than  the  I-Q  frequency.  Either  the  network  low-pass- 
filter  corner  frequency  must  be  so  low  that  there  is  sufficient  attenuation  of 
the  high-frequency  signal  with  only  the  20dB/decade  rolloff  of  a  simple  pole, 
or  else  a  higher-frequency  pole  can  be  introduced  to  strip  the  double-carrier- 
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frequency  component  before  the  r((3xj)  ^  signal  is  low-pass  filtered.  In  either 
case,  the  network  low-pass-filter  corner  frequency  must  be  considerably  lower 
than  the  bandwidth  of  the  components  used  in  the  circuit.  Even  if  no  high- 
frequency  pole  is  explicity  introduced,  there  will  still  be  high-frequency  poles 
where  the  limitations  of  the  multiplier  and  saturating  nonlinearity  bandwidth 
are  reached. 

5.2.3  Automatic  Gain  Control  Approach 

The  automatic  gain  control  approach  for  producing  the  complex  satu¬ 
rating  nonlinearity  is  shown  in  figure  5.2: 


£ 


Figure  5.2:  Automatic  gain  control  approach. 
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It  is  a  nonlinear  feedback  circuit  acting  like  an  automatic  gain  control  for  the 
signal  entering  the  low-pass  filter.  The  advantage  of  this  approach  over  the 
quadrature  oscillator  approach  is  fewer  components,  since  no  oscillator  and 
fewer  multipliers  (only  four  per  unit)  are  required.  However,  some  effort  is 
needed  to  verify  that  the  automatic  gain  control  circuit  serves  as  an  appro¬ 
priate  complex  saturating  nonlinearity,  and  that  as  a  subsystem  it  is  stable 
in  its  own  right. 

The  reason  for  choosing  this  particular  form  for  the  automatic  gain 
control  circuit  is  that  it  uses  only  multipliers,  summers,  and  low-pass  filters 
(the  gain  block  labeled  a  can  be  incorporated  into  the  other  components). 
As  will  be  shown,  the  low-pass  filter  in  the  feedback  loop  is  required  for  the 
automatic  gain  control  loop  to  remain  stable.  The  location  of  the  low-pass 
filter  within  the  feedback  loop  has  been  chosen  to  help  maintain  the  stability 
of  the  overall  network  of  units. 

The  dc  transfer  characteristic  of  the  automatic  gain  control  circuit  will 
now  be  shown  to  satisfy  the  requirements  of  the  function  r(-)  enumerated  in 
section  2.4.  Referring  to  figure  5.2, 

E  j  =  Ej-/  +  a{5  —  \Yjj\2)T,j  (5.1) 

Ej(l  —  0,8  +  o;|Ej|2)  =  Yjjj  (5-2) 
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Sj|(l  -  a;(5  +  «|Sj|2)  —  |Ej/|. 


(5.3) 


Therefore,  we  can  write  r(a)  as  an  implicit  function  of  a: 


r(a)(l  —  aS  +  ar2(a)) 

=  a 

(5.4) 

ar3(a)  +  (1  —  a8)r(a)  —  a 

=  0 

(5.5) 

r3(a)  +  (—  —  S)r(a)  -  — 
a  a 

=  0. 

(5.6) 

The  above  is  a  cubic  polynomial,  which  can  be  solved  as  follows  [27]:  We 
assume  that  (-  —  <$)  >0.  Then  since  —  -  <  0,  we  can  conclude  that  there  is 
a  unique  real  solution  for  r(a).  This  real  solution  is  given  by 


Using  the  binomial  formula  and  the  fact  that  the  composition  of  analytic 
functions  is  analytic,  it  is  easy  to  see  that  this  function  r(-)  is  analytic. 
Furthermore,  r(0)  =  0  and  r(-)  is  strictly  monotone  increasing.  Also,  we  can 
take  !*[  =  or'(a),  with  g(0)  =  constant,  and  for  large  a  approximate: 

a1/3 

r(°)  ~  (5-8) 

We  can  then  further  approximate: 

1  a"2/3 

(5-9) 

dg  1  a1/3 

da  3  a1/3 
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(5.10) 


(5.11) 


9(a) 

9(a) 

r2(a) 


la4/3 
4  a1/3 

itW'3. 


(5.12) 


Thus,  we  may  conclude  that  lim^oo  =  00,  and  hence  that  the  function 
r(-)  produced  by  the  circuit  of  figure  5.2  indeed  leads  to  a  convergent  network 
of  nonlinear  coupled  oscillators. 


Next  we  need  to  show  that  the  automatic  gain  control  subcircuit  viewed 
as  a  nonlinear  feedback  circuit  is  stable.  It  turns  out  this  can  be  shown 
by  a  LaSalle’s  Invariance  Principle  argument  along  the  lines  of  the  proof  of 
Theorem  2.2.  To  simplify  notation  for  the  stability  proof,  the  circuit  of  figure 
5.2  is  redrawn  in  figure  5.3,  with  u  and  z  complex: 


u 


Figure  5.3:  Automatic  gain  control  circuit  for  stability  proof. 


Theorem  5.1 :  The  circuit  of  figure  5.3  converges  to  the  unique  desired 
stable  equilibrium  point  expressed  in  equation  (5.7). 
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Proof.  The  complex  dynamics  are 


—z  =  —  z  +  az(S  —  |z|2)  +  u.  (5.13) 

up 

If  we  let  z  =  pe10  and  u  =  ae1^,  we  can  rewrite  the  dynamics  in  polar 
coordinates  as  follows: 


—  (pe10  +  i8pe10)  =  —pel0  +  apet0(5  —  p2)  +  ae1^  ^ e 10  (5.14) 

up 

— ( p  +  iOp )  =  -p  +  ap(8  -  p2) 

Up 

+<r(cos(,i/>  —  9)  +  i  sin('0  —  9))  (5.15) 

— p  =  —  p  +  ap(S  —  p2)  +  a  cos(ip  —  9)  (5.16) 

LOp 

—p8  =  asin(ijj  —  6).  (5-17) 

Up 

Thus,  provided  p^  0,  we  can  express  the  dynamics  as 


p  =  u)p[-p  +  ap(8  -  p2)  +  a  cos(ip  -  9)) 
6  —  Wp-fcrsin^  —  9)]. 

The  Lyapunov  function  is 


v(p,  9)  =  \p2-a 


pa  cos  (ip  —  9). 


(5.18) 

(5.19) 


(5.20) 


The  Lyapunov  function  is  radially  unbounded  because  of  the  |p4  term  and 
therefore  has  compact  sublevel  sets.  The  derivative  of  the  Lyapunov  function 
along  trajectories  is  computed  as  follows: 


V(p,9) 


dV  .  dV  ■ 
dp  P  +  dd° 


(5.21) 
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(5.22) 


-  —  (— p  +  ap(5  —  p2)  +  acos(ip  —  9)p 

—(pa  sin(^  —  9))9 

=  -up[-p  +  ap(5  -  p2)  +  acos(ip  -  9)]2 

— a;p[ersin(^  —  0)]2.  (5.23) 

Thus,  V (p,  9)  <  0  and  V (p,  9)  —  0  only  if  (p,  9)  is  an  equilibrium  point  of 
the  dynamics,  or  else  p  =  0.  But  if  p  —  0,  then  examination  of  the  original 
complex  dynamics  (5.13)  reveals  that  the  origin  is  only  an  invariant  point 
of  the  dynamics  if  u  =  0,  in  which  case  p  =  0  is  the  unique  desired  stable 
equilibrium  point.  Otherwise,  it  is  clear  from  the  complex  dynamics  (5.13) 
and  the  assumption  that  (^  -  £)  >  0  that  the  unique  equilibrium  point  of  the 
system  has  9  =  • ip  and  p(a)  equivalent  to  r(a)  given  in  (5.7)  for  r(-)  =  p(-) 
and  a  =  a. 

So  by  Theorem  2.1  (LaSalle’s  Invariance  Principle),  and  the  fact  that 
the  dynamics  can  be  written  as  gradiant  dynamics,  we  may  conclude  that 
the  system  (5.13)  converges  to  the  unique  desired  equilibrium  point.  Q.E.D. 

In  practice,  the  circuit  of  figure  5.3  will  have  additional  dynamics  be¬ 
sides  the  low-pass  filter  shown,  since  the  multipliers  have  finite  bandwidth.  If 
there  is  some  frequency  at  which  the  feedback  loop  has  an  overall  gain  greater 
than  unity  but  the  phase  shift  through  the  loop  exceeds  180  degrees,  then 
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clearly  the  circuit  will  be  prone  to  oscillate.  Therefore,  the  pole  frequency 
uip  must  be  chosen  low  enough  that  the  unmodeled  dynamics  do  not  cause 
the  feedback  loop  to  go  unstable. 

5.3  Summary 

Two  methods  of  performing  the  complex  saturating  nonlinearity  func¬ 
tion  for  an  analog  continuous-time  implementation  of  the  network  of  cou¬ 
pled  oscillators  have  been  presented.  The  first,  the  quadrature  oscillator 
approach,  is  conceptually  simple,  but  requires  two  multipliers  per  unit  and 
two  multipliers  per  interconnecting  weight  for  each  nonzero  interconnecting 
weight  in  the  network.  The  second  approach,  the  automatic  gain  control 
circuit,  requires  effort  to  prove  that  it  can  serve  its  purpose,  but  it  requires 
no  oscillator  and  only  four  multipliers  per  unit,  substantially  fewer  than  the 
quadrature  oscillator  approach  even  for  the  relatively  sparcely  interconnected 
circuit  of  figure  4.1.  Either  approach  introduces  a  non-dominant  pole,  either 
explicitly  or  implicitly,  which  in  turn  determines  the  maximum  network  low- 
pass-filter  corner  frequency  for  which  the  overall  network  of  units  will  be 
stable.  Because  of  the  complicated  nonlinear  nature  of  the  network  of  cou¬ 
pled  oscillators  with  additional  dynamics  due  to  the  limited  bandwidths  of 
the  consituent  components,  a  precise  formula  for  determining  the  maximum 
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stable  network  low-pass-filter  corner  frequency  would  be  at  best  very  difficult 
to  derive.  Therefore,  the  choice  of  network  low-pass  filter  corner  frequency 
must  be  made  conservatively,  and  it  should  at  a  minimum  be  verified  that 
with  a  given  dominant  pole  and  non-dominant  pole,  each  pair  of  connected 
units  taken  pairwise  will  not  have  an  obvious  stable  limit  cycle  solution  at 
any  frequency. 
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Chapter  6 


Analog  VLSI  Circuits 

6.1  Introduction 

The  goal  of  this  chapter  is  to  combine  the  practical  issues  of  analog  VLSI 
implementation  discussed  in  the  last  chapter  with  actual  circuit  building 
blocks  in  order  to  explore  tradeoffs  among  network  size,  low-pass-filter  corner 
frequency,  and  power  dissipation.  Using  this  information  the  decision  can 
be  made  as  to  whether  an  analog  VLSI  coupled-oscillator  network  is  worth 
considering  for  a  given  application. 

As  demonstrated  in  the  last  chapter,  analog  multipliers  are  required 
even  for  implementing  the  fixed- weight  dynamics  (2.21)-(2.23),  because  even 
if  the  fixed  weights  are  implementated  as  fixed-gain  amplifiers,  multipliers  are 
still  needed  for  producing  the  complex  saturating  nonlinearities.  Therefore, 
a  multiplier  circuit  is  described  which  could  be  used  with  either  complex 
saturating  nonlinearity  approach  described  in  the  last  chapter,  as  well  as 
to  multiply  the  y fc  by  either  fixed  or  variable  weights  w*k.  Next,  another 
required  circuit  building  block,  the  low-pass  filter,  is  considered,  and  finally 
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the  network  size,  corner  frequency,  and  power  dissipation  issues  are  discussed. 

6.2  Analog  Multiplier  Design 

Among  the  options  for  an  analog  integrated  multiplier  circuit,  I  have 
chosen  to  describe  one  which  is  relatively  simple  to  design,  simple  to  under¬ 
stand,  and  has  a  wide  bandwidth:  the  wide-range  Gilbert  multiplier  of  figure 
6.1: 


Figure  6.1:  Wide-range  Gilbert  multiplier  circuit. 
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This  multiplier  circuit  has  performed  well  in  the  adaptive  filter  project 
of  the  Microelectronic  Device  Physics  Section  at  the  Naval  Research  Lab¬ 
oratory  [21], [23].  The  wide-range  Gilbert  multiplier  circuit  has  the  feature 
that  both  differential  inputs  can  be  biased  at  the  same  dc  voltage  mid-range 
between  the  OV  and  10V  power  supplies,  with  a  couple  of  volts  of  common¬ 
mode  input  range;  therefore,  the  multiplier  is  easy  to  incorporate  into  a 
larger  circuit.  Furthermore,  by  adjusting  the  current  and  W/L  values,  a  va¬ 
riety  of  input  ranges,  gains,  and  bandwidths  can  be  obtained.  The  multiplier 
output  is  a  differential  current,  which  can  be  converted  to  a  differential  or 
single-ended  voltage  by  using  appropriate  load  devices  (shown  in  figure  6.2), 
and  the  outputs  of  several  multipliers  can  easily  be  summed  simply  by  tying 
together  the  output  current  lines  (as  in  figure  6.3). 


DD 


'O* 


•O- 


BIAS 


Figure  6.2:  Multiplier  load  devices  (left:  single-ended,  right:  differential). 


MULTIPLIER  1 


MULTIPLIER  2 


Figure  6.3:  Summing  the  outputs  of  several  multipliers. 

To  demonstrate  how  the  circuit  of  figure  6.1  performs  a  multiplication 
operation,  refer  to  figure  6.4  for  the  following  first-order  analysis  of  the  stan¬ 
dard  Gilbert  multiplier.  All  transistors  are  assumed  to  be  in  saturation, 
where  the  following  (approximate)  equation  holds: 

Id  =  K{Vgs  -  VT )2  (6.1) 

where  K  =  | nC0XW/L ,  with  the  carrier  mobility,  Cox  the  oxide  capacitance 
per  unit  area,  W  the  channel  width,  and  L  the  channel  length.  Ip  is  the  drain 
current,  VGS  is  the  gate-source  voltage,  and  V?  is  the  threshold  voltage  for 
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turning  on  the  transistor.  Transistors  Ml  and  M2  are  assumed  to  be  identical 
with  K  =  Ki,  and  transistors  M3  through  M6  are  assumed  to  be  identical 
with  K  =  K2. 


Figure  6.4:  Standard  Gilbert  multiplier  circuit. 

The  currents  Ii  and  /2  satisfy 

I\  =  Ki(V\  +  Voi)2 
/2  =  K\{— V\  +  Vqi)2 


(6.2) 

(6.3) 
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Is 


(6.4) 


—  h  +  h 

where  Voi  =  \j —  V?  incorporates  the  shift  in  source  voltage  of  Ml  and 
M2  when  the  input  V\  is  applied.  Similarly,  we  can  write  equations  for  the 
currents  /3  through  /6: 


=  K2(V2  +  V03)2 

(6.5) 

=  K2(-V2  +  V03)2 

(6.6) 

=  K2(-V 2  + V05)2 

(6.7) 

=  K2(V2  +  V05)2 

(6.8) 

=  J3+/4 

(6.9) 

=  I5  + 1% 

(6.10) 

where  —  V%  incorporates  the  shift  in  source  voltage  of  M3  and 

M4  when  the  input  V2  is  applied  with  I\  flowing  through  Ml,  and  Vqs  = 


V22  incorporates  the  shift  in  source  voltage  of  M5  and  M6  when  the 


input  V2  is  applied  with  I2  flowing  through  M2.  Noting  that 


V&3  = 

l/zir,  -  * 

(6.11) 

r 

ft™)2- 

V22 

(6.12) 

II 

in 

& 

to 

1 

•3S 

(6.13) 

= 

Vs Kk/~v'  +  v°')2 

-V22 

(6.14) 
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we  can  rewrite  the  equations  for  the  currents  I3  through  Iq  as 


h 

h 

h 

h 


K2 

k2 

k2 

k2 


We  can  now  derive  the  output  current. 


Io  =  I0+-I0-  (6-19) 
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(6.24) 

(6.25) 

(6.26) 


The  final  approximation  is  valid  provided  Vi  is  small  compared  to  V0i ,  Voi  > 
1,  Vi  is  small,  and  Jk  is  modest.  Thus,  the  circuit  does  indeed  approximate 
a  multiplier,  with  the  output  current  as  a  function  of  input  voltage  given  by 

Iq  =  2s/2 T&nm. 


Figure  6.5  shows  how  the  wide-range  Gilbert  multiplier  circuit  compares 
to  the  standard  Gilbert  multiplier  of  figure  6.4.  Transistors  Ml  through  M6 
serve  as  Ml  through  M6  of  figure  6.4.  Transistors  M7  and  M8  form  a  current 
mirror  acting  as  the  current  source  Is  of  figure  6.4.  M9  and  M10  form  a 
current  mirror  to  drive  the  sources  of  M3  and  M4  with  I\  from  Ml.  Similarly, 
Mil  and  M12  form  a  current  mirror  to  drive  the  sources  of  M5  and  M6  with 
I 2  from  M2. 
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Figure  6.5:  Wide-range  Gilbert  multiplier  circuit  for  analysis. 

Deviations  from  the  ideal  multiplier  characteristic  are  caused  by  the 
terms  neglected  in  the  derivation  of  the  multiplier  characteristic,  Early  effect, 
nonideal  characteristics  of  the  current  mirrors,  and  threshold  mismatches 
(offsets)  among  the  multiplier  core  transistors  Ml  through  M6.  Body  effect 
will  also  cause  deviations  from  the  ideal  characteristic,  but  can  be  reduced 
by  putting  as  many  of  Ml  through  M6  as  possible  in  their  own  wells.  For 
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the  circuit  of  figure  6.5  in  a  2  /x m  MOSIS  p-well  technology,  M3-M6  can  be 
placed  in  separate  p-wells. 

6.3  Analog  Low-Pass  Filter  Design 

The  analog  building  blocks  needed  for  the  implementation  of  the  net¬ 
work  of  oscillators  based  on  either  the  quadrature  oscillator  approach  of  figure 
5.1  or  the  automatic  gain  control  approach  of  figure  5.2  for  the  complex  satu¬ 
rating  nonlinearity  include  multipliers,  summers,  low-pass  filters,  and  in  the 
case  of  figure  5.1,  a  real  saturating  nonlinearity  as  well.  Multiplier  design 
has  already  been  discussed,  and  summation  of  currents  at  multiplier  outputs 
has  been  shown  to  not  require  any  extra  circuitry.  (Voltages  can  be  summed 
using  a  circuit  even  simpler  than  the  multiplier  circuit  of  figure  6.4,  so  even 
though  summing  currents  is  trivial,  summing  voltages  is  not  difficult.)  Real 
saturating  nonlinearities  for  the  quadrature  oscillator  approach  can  take  the 
form  of  nonlinear  resistive  elements  (such  as  MOS  diodes  or  p-n  diodes),  or 
can  consist  of  a  small  circuit.  However,  the  low-pass  filter  design  requires 
some  consideration,  because  a  low-pass  filter  can  take  up  much  more  silicon 
real  estate  than  a  multiplier.  Furthermore,  there  are  constraints  on  what 
approaches  can  be  used  to  achieve  various  time  constants. 

As  a  starting  point  for  design  of  a  low-pass  filter  in  MOSIS  2  /xm 
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double-polysilicon  p-well  technology,  we  will  take  the  maximum  size  of  a 
poly-poly  capacitor  (a  subjective  decision  based  on  keeping  the  capacitor 
size  on  roughly  the  same  scale  as  two  multipliers)  to  be  3pF.  Using  a  p-well 
resistor,  resistances  up  to  about  3MQ  can  be  achieved  in  a  reasonable  area 
for  this  application.  Thus,  a  direct  RC  time  constant  down  to  about  20  kHz 
can  be  achieved. 

Another  approach  for  implementing  the  low-pass  filter  is  to  use  the 
transconductance-C  circuit  of  figure  6.6.  The  time  constant  for  this  circuit  is 
approximately  given  by  gm/C  where  gm  is  the  small-signal  transconductance 
of  M3  and  M4.  An  advantage  of  the  transconductance-C  circuit  over  the 
direct  RC  circuit  is  that  the  transconductance-C  circuit  can  provide  voltage 
gain.  The  circuit  of  figure  6.6  can  be  used  to  generate  time  constants  down 
to  the  10s  of  kHz  range. 
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Figure  6.6:  Transconductance-C  low-pass  filter. 
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The  circuit  of  figure  6.7,  known  as  a  transconductance-killer  circuit 
because  the  cross-coupled  input  stage  acts  to  reduce  the  overall  transcon¬ 
ductance  below  that  of  the  individual  transistors,  has  been  used  to  produce 
time  constants  down  to  1  kHz.  For  lower  corner  frequencies,  continuous¬ 
time  approaches  tend  to  require  too  much  chip  area,  and  switched-capacitor 
circuits  should  be  considered. 


Ml 


v  -  10  V 
DD 


M2 


■  200  UA 


M3 

i 

e— 

n 

1 

IN 


W/L 

Ml  -  M2  4/40 

M3  -  M4  4/80 

M5  -  M6  4/76 

M7-M10  4/100 


M5  M6 


M7 


M4 


M8 


vOUT 


M9 


M10 


V  =  0  V 

_  COM  u 


Figure  6.7:  Transconductance- killer  low-pass  filter. 


Of  course,  the  low-pass  filter  design  must  be  a  simple  pole.  Any  pro¬ 
posed  low-pass  filter  design  must  be  analyzed  or  simulated  to  make  sure  it  will 
behave  as  a  single-pole  filter  out  to  a  high  enough  frequency.  P-well  resistors 
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have  distributed  capacitance,  and  the  transconductance-C  circuits  of  figures 
6.6  and  6.7  have  additional  dynamics  which  must  be  taken  into  account.  Ob¬ 
viously  the  low-pass  filters  just  listed  are  by  no  means  the  only  ones  which 
could  be  considered.  For  example,  an  active  op-amp  low-pass  filter  can  trade 
off  increased  power  dissipation  to  produce  a  more  ideal  single-pole  filter. 

6.4  Tradeoffs  among  Power  Dissipation,  Network  Low- 
Pass-Filter  Corner  Frequency,  and  Physical  Circuit  Size 

The  purpose  of  describing  the  multiplier  circuit  at  length  and  of  intro¬ 
ducing  some  low-pass-filter  circuits  is  that  they  enable  tradeoffs  among  power 
dissipation,  network  low-pass-filter  corner  frequency,  and  physical  circuit  size 
to  be  explored.  This  is  a  first  step  toward  determining  potential  applications 
for  analog  implementations  of  such  networks. 

To  consider  the  tradeoff  between  power  dissipation  and  network  low- 
pass-filter  corner  frequency,  some  assumptions  are  needed.  First,  it  will  be 
assumed  that  the  multipliers  dominate  the  power  dissipation.  Second,  the 
total  power  dissipation  allowed  will  be  taken  to  be  1  W  (roughly  the  power 
level  at  which  in  a  room  temperature  environment  without  heat  sinking  the 
package  will  feel  warm  to  the  touch,  but  not  actually  hot  enough  to  burn  a 
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finger).  Finally,  for  the  circuit  of  figure  6.1,  with  Is  =  200//A  corresponding 
to  a  power  dissipation  of  4  mW  per  multiplier,  the  bandwidth  at  which 
“slight”  deterioration  of  the  frequency  response  is  seen  to  occur  is  taken  to 
be  20  MHz  (multiplier  circuits  cannot  be  characterized  by  a  single  3dB  point, 
because  the  frequency  response  is  different  for  the  two  inputs).  This  estimate 
is  based  on  testing  larger  adaptive  filter  circuits  of  which  the  multiplier  of 
figure  6.1  was  a  subcircuit. 

As  the  multiplier  current  is  decreased,  its  bandwidth  will  also  decrease, 
but  because  reducing  the  current  level  requires  changing  the  transistor  sizes 
in  order  to  maintain  the  same  input  signal  range,  the  relationship  between 
multiplier  current  and  bandwidth  can  only  be  understood  with  the  aid  of  the 
derivation  of  the  multiplier  characteristic  given  in  section  6.2.  The  relation¬ 
ship  between  current  and  bandwidth  can  be  estimated  by  taking  bandwidth 
to  be  proportional  to  gm,  which  depends  both  on  transistor  W/L  and  on 
drain  current  according  to  gm  =  2 \JKIp  =  ^j2fiCoxj-Io-  But  in  order  for 
the  approximation  (6.26)  to  be  valid,  Voi  =  —  V?  must  remain  constant 

as  Is  is  decreased;  therefore,  K\  must  decrease  in  proportion  to  Is-  But  to 
keep  the  multiplier  gain  constant,  K2  must  then  also  decrease  in  proportion 
to  Is-  Hence  the  W/L  of  all  the  multiplier  core  transistors  must  be  reduced 
in  proportion  to  the  reduction  in  Is  in  order  to  maintain  the  same  range 
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and  gain.  Therefore,  since  K  is  proportional  to  Id  and  gm  is  proportional  to 
\/KId,  gm  is  proportional  to  Id-  Thus,  for  the  multiplier  circuits  of  figures 
6.4  and  6.5,  bandwidth  reduction  is  approximately  linear  with  decreasing 
current. 

Changing  W/L  will  have  an  effect  on  transistor  capacitances  as  well 
as  on  gm,  but  the  bandwidth  changes  due  to  decreasing  Id  (using  the  4mW 
multiplier  as  a  starting  point)  will  be  primarily  due  to  the  change  in  gm. 
However,  as  Id  is  increased,  there  comes  a  point  where  due  to  Early  effect  or 
short-channel  limitations  L  cannot  be  decreased  any  further  so  that  increasing 
W/L  means  increasing  only  W.  Overlap  capacitances  increase  linearly  with 
W,  so  increasing  current  increases  both  gm  and  the  transistor  capacitances. 
Therefore,  instead  of  bandwidth  increasing  linearly  with  ID,  it  will  increase 
only  as  the  square  root  of  Id-  Moreover,  finite  driving  impedances  and 
Miller  effect  act  to  prevent  further  increase  in  bandwidth  with  increasing 
current  after  a  certain  point.  The  20MHz  multiplier  is  probably  near  the 
point  of  diminishing  returns  so  that  increasing  multiplier  power  above  4mW 
for  the  wide-range  Gilbert  configuration  would  not  substantially  improve  its 
bandwidth  performance. 

If  we  assume  a  sparcely  interconnected  network  of  coupled  oscillators, 
and  assume  four  multipliers  per  unit  are  used  to  effect  the  complex  saturating 
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nonlinearity  while  an  average  of  four  more  (real)  multipliers  per  unit  are 
used  for  the  complex  interconnecting  weights,  then  with  a  maximum  power 
dissipation  of  1  W,  at  4mW  per  multiplier  we  can  build  roughly  a  30-unit 
network. 

It  is  more  difficult  to  give  a  clear-cut  tradeoff  between  number  of  mul¬ 
tipliers  and  total  silicon  real  estate  than  was  the  case  for  power  dissipation. 
This  is  because  low-pass  filters  tend  to  require  more  area  than  multipliers 
due  to  the  capacitors,  and  interconnections  can  also  consume  a  significant 
amount  of  area  for  a  large  number  of  units.  For  the  adaptive  filter  project, 
two  multipliers  and  two  low-pass  filters  with  3pF  capacitors  (accompanied 
by  switching  and  biasing  circuitry)  were  integrated  in  a  250/mi  x  1185//m  cell 
size.  Therefore,  1mm2  per  unit  would  be  a  reasonable  estimate  of  space  re¬ 
quired.  A  30-unit  network  would  therefore  fit  on  a  6mm  x  6mm  chip,  which 
can  be  accomodated  by  MOSIS. 

Now  that  the  tradeoffs  between  component  bandwidth,  power  dissipa¬ 
tion,  and  circuit  size  have  been  given  some  perspective,  the  relationship  be¬ 
tween  component  bandwidth  and  network  low-pass-filter  corner  frequency  is 
all  that  remains  to  describe.  There  are  several  factors  which  need  to  be  taken 
into  account,  because  how  far  the  automatic  gain  control  loop  pole  must  be 
below  the  multiplier  bandwidth  depends  on  the  gain  of  the  automatic  gain 
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control  loop,  and  how  far  the  network  low-pass-filter  corner  frequency  must 
be  below  the  non-dominant  network  pole  depends  on  the  gain  of  the  function 
r(-)  near  zero  and  on  the  weight  values.  A  very  rough  estimate,  for  purposes 
of  determining  potential  applications  only,  is  that  the  non-dominant  network 
pole  (or  the  automatic  gain  control  loop  pole)  should  be  down  a  decade 
from  the  frequency  at  which  the  multiplier  begins  to  show  “slight”  frequency 
response  deterioration,  and  the  dominant  pole  (the  network  low-pass-filter 
corner  frequency)  should  be  down  a  decade  from  the  non-dominant  pole  fre¬ 
quency.  Thus,  the  4mW  multiplier  would  give  roughly  a  200kHz  network.  Of 
course,  with  good  simulation  tools  and  higher-speed  processes  than  MOSIS, 
much  higher  bandwidths  could  ultimately  be  obtainable. 
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Chapter  7 


Conclusions  and  Recommendations  for  Future 
Work 

To  summarize,  convergence  results  for  a  class  of  networks  of  nonlin¬ 
ear  coupled  oscillators  (or  directional  units,  depending  on  the  point  of  view 
taken)  have  been  presented.  Physical  motivation  has  been  given  for  the 
simplest  network  considered,  but  the  convergence  results  for  the  rest  of  the 
networks  are  based  solely  on  the  adaptability  of  the  original  Lyapunov  func¬ 
tion  arguments  to  the  various  networks.  Also,  a  simple  example  circuit  il¬ 
lustrates  that  such  networks  can  be  used  in  feedback  configurations,  with 
convergence  properties  that  can  be  deduced  simply  if  the  circuit  design  fol¬ 
lows  certain  rules.  In  particular,  two  distinct  approaches  for  eliminating 
undesired  stable  equilibria  due  to  unit  amplitudes  going  to  zero  were  pre¬ 
sented.  Next,  analog  VLSI  implementation  considerations  were  discussed  for 
such  networks.  Tradeoffs  among  power  dissipation,  bandwidth,  and  network 
size  were  presented  to  assist  in  determining  potential  applications  for  analog 
VLSI  networks  of  coupled  oscillators. 

One  contribution  of  this  work  was  to  point  out  that  the  dynamics  (1.1)- 
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(1.2)  converge  as  a  consequence  of  the  convergence  properties  of  the  dynamics 
(2.12).  One  direction  for  future  work  would  be  to  undertake  a  study  of  to 
what  extent,  if  at  all,  other  types  of  coupled  oscillator  networks  can  be  made 
to  conform  to  the  convergence  results  presented  here.  In  particular,  one 
could  attempt  to  extend  the  convergence  results  to  the  situation  where  the 
oscillators  do  not  share  a  common  frequency  for  all  time. 

Another  direction  for  future  work  could  consist  of  developing  more  gen¬ 
eral  techniques  (than  Theorems  4.1  and  4.2)  for  determining  the  equilibrium 
points  and  whether  they  are  stable.  Then  the  dependence  of  the  equilibria 
upon  external  weight  controls  could  be  specified  so  that  the  stable  equilibria 
of  the  network  could  be  programmed  using  control  inputs.  An  example  of 
a  situation  where  this  might  be  useful  would  be  in  smoothly  transitioning 
between  different  gaits  in  a  walking  robot. 

Finally,  there  is  an  interesting  implication  of  the  adaptively  controlled 
network  convergence  result  in  terms  of  hierarchical  control  of  coupled  oscilla¬ 
tor  networks.  If  two  or  more  networks  are  arranged  in  a  hierarchical  fashion, 
with  the  units  of  one  network  serving  as  the  weights  for  the  next  network 
in  the  hierarchy,  the  convergence  result  implies  that  feedback  can  be  applied 
from  lower  levels  to  higher  levels  and  stability  will  be  maintained,  as  long  as 
the  adaptive  control  law  is  obeyed.  Consideration  of  networks  of  coupled  os- 
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dilators  for  hierarchical  control  systems  with  local  and  global  feedback  might 
therefore  be  worthwhile. 
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Appendix  A 


Calculation:  j3ajr(j3aj)  —  log  (2ttIq (/?%))  — >  oo  as  dj  — >  oo 


To  summarize  the  calculation,  first,  we  let  m  =  /3aj,  and  rewrite  the 
expression  ignoring  the  constant  term  —  log  2ir. 

mr(m )  —  lnJo(m)  =  (A.l) 

Clearly  pa^ipaj)  —  log  (2tt/o(/3%))  — y  oo  as  aj  — >  oo  if  and  only  if  /i(m)  — >• 
oo  as  m  -»  oo.  Next,  we  split  /i(m)  into  two  terms,  as  follows: 

fi{m)  =  mr(m)  -  In J0(m) 

=  (m  —  \nl0{m))  —  m  (1  —  r(m))  (A. 2) 

=  him)  -  him). 

Then  we  show  that  him)  — >  oo  as  m  — >  oo  and  him)  <  1  Vm.  Then  we 
may  conclude  that  /i(m)  -4  oo  as  m  — >  oo. 


Now  we  show  that  him )  ~  m  —  ln/0(m)  =  mine  —  In  him)  = 
—  In e_m/0(m)  — >■  oo  as  m  — >  oo.  The  asymptotic  behavior  of  Jo(m)  as 
m  — >•  oo  is  given  by  [28]: 


him) 


V2 


ion 


_ 9_ 

8m  2!  (8m)2 


(A.3) 
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Thus,  for  m  large  enough,  e  m/0(m)  behaves  like  ■ 


We  can  proceed  rigorously  as  follows:  For  M  sufficiently  large,  we  have 
for  m>  M, 


e~mh  (m)  < 
1 


e~mI0(m) 

fi(m)  =  In  r 

e  mIo(m) 


> 


\phvm 

\j2ixm 


(assume  M  >  1) 


7r  m 

~Y 


,  /7rm  1,  to  , . 

>  In  i/  ——  =  -  m  — - ^ooasm-^oo.  (A. 4) 

z  z  z 


Thus,  /2  (m)  — >  oo  as  m  — >  oo. 


Next  we  show  that  /3(m)  —  m(  1  —  r(m))  <  1  Vm. 


AH 


dh 

dm 


m  (1  —  r(m)) 

h(m)\ 
Io(m) ) 
h  (m) 
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/0(m); 


+  m 


^-(/1(m))/0(m)  +  7i2(m)' 
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702(m) 


(note:  ^ 


1  - 


AM  _  m±(h{m)) 
h(m) 
him) 


+  m 


h{m) 
m!o(m )  —  Ji(m) 


7o  (to)  /0(m) 

1  —  m  +  mr2(m ) 


+  m 


/oH/ 

7i(m) 
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=  1  —  m  (l  —  r2(m))  . 


(A-5) 
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Observe  that: 


r(m) 

< 

1 

r2(m) 

< 

r(m) 

1  —  r2(m) 

> 

1  —  r(m) 

—  r2(m)) 

> 

m  (1  —  r(m 

dm 

> 

/sM 

dfz 

dm 

< 

1  -  /sH. 

(A.6) 


Let  / 4(771)  be  defined  by: 


dh  1  - ,  v 

^  = 1  - /4(m) 

/4(0)  =  /3(0)  =  0. 


(A.7) 


Then  by  a  simple  integration,  /4(m)  =  1  -  e~m.  Since  /3( 0)  =  /4(0),  /3(m) 
is  continuously  differentiable,  and  ^  ^  Vm,  it  follows  that 


/3(t7i)  <  /4(m)  =  1  -  e  m  <  1  Vm. 
Thus,  /3(m)  <  1  Vm. 


(A.8) 


By  having  shown  /2(m)  ->■  00  as  m  ->•  00  and  /3(m)  <  1  Vm,  we  may 
now  conclude  that,  indeed, 


Pajr{Paj)  -  log  {2vh{fiaj))  -»■  00  as  a,j  — >•  00. 


(A.9) 
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Appendix  B 


Convergence  property  shared  by  fixed-weight  networks 

Two  sets  of  dynamics  for  fixed-weight  networks  of  coupled  oscillators 
were  presented.  The  first  set  was  (2.21)-(2.23),  repeated  below: 


Vi  =  -Vi  +  r(P\xi\)PL\ 

(B.l) 

N 

x3  =  Y  y*wjk 

(B.2) 

k= 1 

X  * 

at  equilibrium,  yj  =  r((3\xj\)-~. 

\xj\ 

(B.3) 

The  second  set  was  (2.24)-(2.26),  repeated  below: 


Xj  =  -Xj  +  Y,  VkWjk 

k 

(B.4) 

& 

II 

5- 

(B.5) 

n 

at  equilibrium,  Xj  =  ^  Vkwjk 

(B.6) 

k= 1 

(this  is  the  network  of  Zemel  et.  al.) 


The  convergence  proof  of  section  2.3  applies  directly  to  the  second  set  of 
dynamics;  thus,  the  Xj  and  yj  variables  in  the  second  set  of  dynamics  converge 
to  an  equilibrium  point.  As  shown  by  equation  (2.3),  the  Xj  variables  in 
the  first  set  of  dynamics  satisfy  the  same  differential  equation  as  the  xD 
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variables  in  the  second  set  of  dynamics.  Therefore,  all  that  needs  to  be 
proved  to  conclude  that  the  first  set  of  dynamics  converges  to  an  equilibrium 
point  is  that  Xj(t )  — »  Xj  as  t  — »  oo  with  yj  =  —yj  +  r((3\xj\)-^  implies 
yj(t)  — y  y*  =  r(f3\x*\)j^  as  t  — >■  oo.  This  will  follow  from  the  following 
lemma: 


Lemma  B.l:  Let  ^  =  —u  +  f(t),  where  limt_foo/(t)  =  0.  Then 
lim^oo  u(t)  =  0. 

Proof.  Observe  that  if  |/(f)|  <  e  \/t  >  T,  then  for  t  >T, 


u(t)  =  e  t^u(T)  +  J  e  T)/(r)dT 

(B.7) 

\u(t)\  <  \u(T)\  +  e  f  e~^~TW 

JT 

(B.8) 

=  e-^\u(T)\  +  e(l-e~^) 

(B.9) 

<  e~^\u(T)  |  +  e. 

(B.10) 

So  given  ex,  we  can  find  T\  such  that  \f(t)\  <  ^  V£  >  T\. 

For  any  initial 

condition  u( 0),  we  can  compute 

w(Ti)  =  e-Tlu(0)  +  J  e-(Ti-T) /(r)dr 

(B.ll) 

and  then  take  T2  >  T\  large  enough  that  e  ^  Tl)|tt(Ti)|  <  Then  Vt  >  T2, 
\u(t)\  <  t\.  Hence,  limt^oou(t)  =  0.  Q.E.D. 
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To  apply  this  lemma,  let  u(t)  =  yj(t)-r(/3\x*\)-^  and  f(t)  =  r(/3 \xj\)^ 
r((3\x*j\)0r\.  Then  u(t)  satisfies  u(t)  =  —  u(t)  +  f(t),  and  f(t)  — >  0  as  t  — »  oo. 

X* 

Hence,  u(t)  — >  0  as  t  — >■  oo,  and  we  may  conclude  that  yj(t )  — >  r((3 |^|)|ji| 
as  t  — »  oo. 
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Appendix  C 


Derivation  of  the  Implementable  Version  of  the  Adap¬ 
tively  Controlled  Network 


Vj  =  -Vj+r(P 

(C.l) 

xj  =  J2ykwjk  +  Uj 
k=  1 

(C.2) 

jj  1  1  :  Wiuh  =  diuiAP\xj\)§-\ 

11,12  \  0  :  otherwise. 

(0.3) 

Then 

xj  =  Y,ykW*k  +  ^2ykw*jk  +  Uj 
k  k 

=  E  VkW*k  +  E  fyhVh +  Uj 

k  l\,l  2 

=  E  [-Vk  +  r(P\xk\) w*k  +  Y,  IjjhVhWjjb  +  Uj 
=  ~xi  +  E  +  E  ^hVh ™*j,i2  +  Uj  +  Uj.  (C.4) 

k  \Xk\  h,l2 


If  we  define  the  dynamics  of  Uj  as 


Ui 


-Uj  -  E 


2Wj,h 


h,h 


+  'EIhhdh,l2r{P\xi1\)r{P\xl2\) 
h,h 


%l2  Xh 

M  K 


(C.5) 
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we  obtain 


ij  =  -xj  +  Er(/3 \xk\)^-w*jk  +  tfi ,h dii ,hr(0\xh\)r(P\xi2\) ry^T • 

k  l*cfel  h,l2  \Xl 2 1  l^i  I 

(0.6) 

Because  we  wish  to  rewrite  the  dynamics  in  a  form  homogeneous  in  the  x 
variables,  let 

=  +  (C.7) 

luh 

Then 


Wj 


=  Pi  +  Ete  +  E^ 


ih 


=  ~u3  +  E  +  E  Illh^wlh 

=  ~W3  +  E 

+  E  4/2^2 +  y;2K,2 

l  i,h 

=  -Wf  +  E  //iAd<i^r(^lg<il)r(^la;fal) 

+  E/S2r’(^l^l)^T<(2 

h,l2  lX<2l 

=  -W)  +  £  +  W  r(/?K|)r(/?|x,J)i^|^.  (C.8) 


h,h 


Thus,  the  implementable  dynamics  are 

%  =  -^  +  K/?N)rrr  (°-9) 

=  E  Vkw*jk  +  Wj-J2  ^h^hVhrWK |A  (C.10) 

fc=l  ii,J2  \xh\ 
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Wj  =  -Wj  +  E  (tU 

h,h 


+  Ij1,l2dih)rW\xh\)r(P\xh\) 


xh 

M  M 


(C.ll) 


at  equilibrium,  yj  =  r{(3\xj\) 


(C.12) 
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